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Executive  Summary 


Assuring  the  future  affordability  of  the  Department  of  Defense’s  (DOD)  portfolio  of 
acquisition  programs  has  been  an  enduring  goal  that  has  rarely  been  achieved.  The  consequences 
of  failure  are  cancelling  or  curtailing  programs  that  turn  out  to  be  unaffordable,  with  attendant 
waste.  This  research  is  a  sequel  to  previous  research  by  the  Institute  for  Defense  Analyses  (IDA) 
on  the  affordability  of  defense  acquisition  programs.^  The  previous  research  focused  on  the 
context  and  theory  of  assessing  affordability.  The  conclusions  and  recommendations  of  the 
previous  publications  are  re-enforced  by  the  current  research,  which  focuses  on  two  areas:  (1) 
what  can  be  learned  from  historical  efforts  to  control  the  affordability  of  DOD  programs,  and  (2) 
how  affordability  constraints  for  sustainment  costs  can  best  be  implemented. 

The  Better  Buying  Power  (BBP)  initiatives  mandate  “affordability  as  a  requirement”  by 
setting  affordability  targets  for  both  investment  and  sustainment,  or  operating  and  support 
(O&S),  costs  to  be  considered  equivalent  to  key  performance  parameters.^ 

Historical  Efforts  to  Control  Affordability 

Two  previous  acquisition  initiatives  bear  sufficient  similarities  to  the  current  affordability 
initiatives  to  provide  meaningful  insights.  They  are  the  Design  to  Cost  (DTC)  efforts  of  the  early 
1970  to  early  1992  period  and  the  Cost  as  an  Independent  Variable  (CAIV)  initiatives  in  1996- 
2002. 

The  research  team  reviewed  the  major  DOD  issuances  that  were  in  force  for  both  the  DTC 
and  CAIV  eras  to  identify  provisions  that  were  similar  to  current  affordability  policy.  This 
review  revealed  a  great  deal  of  similarity,  as  well  as  some  key  differences,  between  both  DTC 
and  CAIV  and  current  policy.  The  most  significant  difference  is  that  in  some  of  their 
manifestations  both  DTC  and  CAIV  stressed  aggressive  efforts  to  reduce  costs  (focused  on 
investment),  and  thus  were  more  similar  to  the  should-cost  initiative  of  BBP.  Those  provisos 
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were  based  on  the  questionable  premise  that  commercial  developments  were  far  more  successful 
in  driving  unnecessary  costs  out  of  products  in  development  than  was  DOD. 

DTC  was  “on  the  books”  for  about  twenty  years;  however,  by  the  late  1980s  it  had  fallen 
into  disuse.  Nonetheless,  according  to  IDA  research  from  the  late  1980-early  1990  period,  some 
thirty-four  DOD  acquisition  programs  recorded  DTC  goals  in  their  Selected  Acquisition  Reports 
(SAR)  between  1971  and  1990  while  forty-seven  programs  of  the  same  era  did  not.  The  previous 
IDA  research  found  that  programs  using  DTC  had  significantly  higher  cost  growth  than  those 
that  didn’t,  as  shown  in  the  following  table. 


Comparison  of  Production  Cost  for  Programs  Using  DTC 


Programs  (34)  Using  DTC 

Programs  (47)  Not  Using  DTC 

Mean  Production  Cost  Growth 

85% 

61% 

Median  Production  Cost  Growth 

66% 

22% 

The  start  dates  of  non-DTC  programs  were  generally  earlier  than  the  start  dates  of  DTC 
programs  (median  start  dates:  1970  vs.  1976).  Starting  in  the  late  1970s,  numerous  changes 
occurred  in  acquisition  policies  and  program  management  practices,  so  it  cannot  be  concluded 
that  using  a  DTC  strategy  was  actually  the  cause  of  the  observed  cost  growth. 

Perhaps  of  greater  significance  than  the  quantitative  results  are  the  findings  based  on 
interviews.  While  the  research  team  did  not  interview  a  large  number  of  individuals  from  the 
DTC  era,  those  interviewed  were  in  key  positions  to  understand  the  degree  to  which  DTC  was 
actually  used  in  acquisition  management.  The  interviewees  reported  that  they  could  recall  no 
instances  when  establishing  or  exceeding  DTC  goals  was  a  topic  of  high-level  deliberations. 
Review  of  several  key  decision  documents  of  the  period  also  revealed  no  discussion  of  DTC. 
Although  by  no  means  conclusive,  this  limited  evidence  suggests  that,  despite  the  strong 
language  regarding  DTC  in  DOD  issuances,  the  concept  may  not  have  been  consistently  stressed 
by  top  acquisition  and  program  managers. 

CAIV  was  in  use  for  a  much  shorter  time  than  DTC;  nonetheless,  it  was  well  entrenched  in 
DOD  regulations  from  1996  to  2002^  and  received  the  attention  of  high-level  management  for 
several  years.  Unlike,  DTC,  however,  CAIV  cost  objectives  were  never  systematically  recorded 
in  SARs,  and  for  that  reason,  among  others,  no  database  exists  to  support  statistical  analysis  of 
CAIV’s  impact.  Since  constructing  such  a  database  was  beyond  the  scope  of  this  paper,  the 
research  team  conducted  an  analysis  of  twelve  programs  known  to  have  employed  the  CAIV 
approach.  The  results  were  that  only  two  of  the  twelve  CAIV  programs  reviewed  could  be  said  to 


The  observed  difference  was  statistically  significant  at  the  7  percent  level  using  a  Kolmogorov- Smirnov  test. 
It  still  exists  in  a  vestigial  form  in  DODD  5000. 1  (published  in  2007)  and  is  still  current. 
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have  been  fully  sueeessful  in  eontrolling  cost  growth.  The  Joint  Direct  Attack  Munition  (JDAM) 
program  was  successful  in  not  only  controlling,  but  also  significantly  reducing,  production  costs 
and  was  in  fact  one  of  the  inspirations  for  CAIV.^  Compared  to  all  the  other  programs  examined, 
JDAM  was  notable  for  its  simplicity — it  was  an  ideal  CAIV  candidate. 

The  failure  of  CAIV  to  control  system  cost  growth  offers  lessons  for  DOD’s  current  efforts. 
Although  each  program  has  a  unique  story,  the  common  thread  is  that  each  program  was  started 
based  on  assumptions  {i.Q.,  framing  assumptions)  that  subsequently  did  not  hold  up.  Of  the  ten 
CAIV  programs  that  were  unsuccessful  in  controlling  costs,  two  were  cancelled  (but  probably 
not  primarily  because  of  cost  growth).  The  others  were  deemed  of  such  importance  that  they 
were  restructured — some  repeatedly — and  continued  under  new  (usually  implicit)  framing 
assumptions. 

DTC,  CAIV  and  the  current  affordability  goals  and  caps  all  depend  ultimately  on  the 
concept  of  controlling  costs  by  trading  performance  characteristics  (and  to  a  much  less  extent, 
schedule).  This  approach  will  only  be  implementable  if  there  is  enough  performance  trade  space 
to  be  exploited.  In  some  of  the  CAIV  programs  reviewed,  that  was  not  the  case.  In  others,  the 
available  trade  space  was  insufficient  to  compensate  for  the  cost  growth.  The  lesson  is  that  the 
amount  of  performance  trade  space  that  is  available  should  be  considered  in  setting  affordability 
targets.  If  little  is  available,  a  greater  tolerance  for  cost  growth  may  have  to  be  accepted  if  the 
capabilities  offered  by  a  new  program  are  truly  required. 

Affordability  of  O&S  Costs 

Assessing  the  O&S  affordability  of  a  proposed  new  system  means  considering  an 
appropriate  portfolio  of  future  force  O&S  costs  that  include  the  new  program  to  make  a  realistic 
projection  of  whether  sufficient  funds  will  likely  be  available  for  the  portfolio.  The  definition  of 
the  portfolio  of  forces  must  be  made  so  that  the  O&S  cost  change  occurring  with  the  deployment 
of  the  new  system  is  clear.  Current  policy  states  that  programs  that  are  seen  as  not  affordable 
based  on  life-cycle  costs  should  not  be  allowed  to  proceed  unless  program  or  programmatic 
adjustments  can  be  identified  to  make  it  affordable.  Such  assessments  are  complicated. 

A  simpler  way  to  assess  O&S  cost  affordability  is  to  compare  the  estimated  O&S  cost  of  a 
new  system  to  the  system  or  systems  that  it  will  replace — known  as  antecedent  systems.  Systems 
with  O&S  costs  significantly  greater  than  the  systems  they  will  replace  may  not  be  affordable  to 
operate  in  the  future.  The  U.S.  Congress  has  required  that  such  a  comparison  be  included  in 
SARs  when  an  appropriate  antecedent  system  exists.  However,  only  forty-two  out  of  a  total  of 
eighty-three  programs  having  a  SAR  in  December  2012  provided  a  comparison  of  O&S  costs  to 
an  antecedent  system.  For  those  forty-two,  the  average  change  relative  to  the  antecedent  was  +13 


Also  an  inspiration  for  the  notion  that  aggressive  action  on  the  part  of  program  managers  could  achieve 
substantial  cost  reductions,  in  line  with  the  should-cost  approach.  Some  of  the  early  CAIV  literature  spoke  of 
achieving  a  25  percent  reduction  from  “expected”  costs — i.e.,  standard  program  cost  estimates. 
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percent,  indicating  a  potential  affordability  problem  in  O&S  costs  when  these  systems  are 
fielded.^ 

Another  issue  in  managing  O&S  cost  affordability  constraints  is  the  uncertainty  in  O&S 
cost  estimates  for  new  systems.  The  uncertainty  can  be  expected  to  decrease  as  systems  mature; 
however,  it  will  still  be  high  at  least  until  the  system  completes  initial  operational  tests.  The  true 
costs  will  not  be  known  until  well  after  fielding.  Since  there  is  a  tendency  to  underestimate  future 
O&S  costs  early  in  the  acquisition  process,  there  is  the  danger  that  O&S  affordability  constraints 
will  be  set  unrealistically  low.  Comparisons  of  O&S  cost  estimates  over  time  show  that  O&S 
cost  estimates  increased  from  pre-full-scale  development  to  early  production  for  virtually  every 
program  examined.  Changes  ranged  from  +50  percent  to  over  +100  percent  from  earliest 
estimates  to  current  estimates.  To  be  enforceable,  O&S  affordability  goals  and  caps  must  be 
supported  by  realistic  estimates.  O&S  costs  over  time  should  be  re-examined  at  least  annually  to 
prevent  a  big  “surprise”  increase  at  the  decision  to  enter  production. 

Meeting  O&S  cost  goals  will  most  likely  require  tradeoffs  to  improve  the  O&S  cost  drivers. 
Early  tradeoffs  (e.g.,  before  Milestone  B)  are  much  easier  to  implement  than  later  in  the  process. 
However,  for  Major  Defense  Acquisition  Programs,  there  is  no  body  empowered  to  oversee  such 
tradeoffs  prior  to  Milestone  B. 

Implementation  of  a  standard  O&S  cost  model  for  each  program  passing  Milestone  A  could 
greatly  facilitate  both  tracking  O&S  costs  as  the  system  matures  and  making  tradeoffs  to  ensure 
that  projected  O&S  costs  remain  within  affordability  constraints.  Such  a  model  should  include 
the  major  controllable  O&S  cost  drivers.  The  model  would  be  refined  as  the  system  matures  and 
better  estimates  of  the  drivers  become  available.  Furthermore,  the  model  should  be  applied  to 
both  the  new  system  and  its  antecedents,  as  well  as  other  systems  within  the  portfolio. 

Conclusions 

•  Experience  with  the  DTC  and  CAIV  initiatives  indicates  that  past  efforts  to  control 
costs  to  ensure  the  affordability  of  DOD  acquisition  programs  have  been  largely 
ineffective  despite  having  strongly  worded  policies  in  place.  Probable  causes  are  lack  of 

-  sufficient  emphasis  on  the  critical  early  phases  of  program  concept  formulation  and 
systems  engineering,  and 

-  top  level,  consistent,  and  persistent  management  attention  and  effective  processes 
for  periodic  re-assessment  of  program  affordability. 


Potential  because  per-system  increases  might  be  mitigated  by  deploying  fewer  systems  or  perhaps  offset  by 
reductions  elsewhere  that  can  be  taken  because  of  the  new  system’s  capabilities. 

Changes  in  the  ground  rules  for  O&S  cost  calculation  in  2010  required  a  manual  adjustment  for  each  program  in 
order  to  track  O&S  cost  growth,  which  limited  the  number  of  programs  for  which  the  comparisons  could  be 
made. 
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•  Affordability  goals  and  caps  established  early  in  a  program  will  only  be  relevant  so  long 
as  the  framing  assumptions  under  whieh  they  were  established  eontinue  to  hold. 
Furthermore,  it  is  inevitable  for  the  assumptions  to  ehange,  sometimes  greatly.  Thus  a 
proeess  is  needed  to  periodieahy  re-examine  the  affordability  of  the  entire  investment 
program  and  make  the  neeessary  adjustments  needed  to  stay  on  an  affordable  traek. 

•  O&S  eosts  have  been  largely  ignored  in  post-Milestone  A  systems  design  trades  in  the 
past. 

-  Controlling  longer-term  O&S  eosts  is  substantially  more  difficult  than  nearer-term 
investment  eosts  beeause  of  the  uneertainty  in  O&S  eost  estimates,  particularly 
early  in  the  aequisition  proeess. 

-  It  is  difficult  to  motivate  aequisition  managers  and  eontractors  to  maintain  control 
over  O&S  eosts. 

•  Reeently  introduced  proeesses  and  proeedures  to  monitor  O&S  costs  and  affordability 
appear  to  be  on  the  right  traek,  but  have  yet  to  be  proven  effeetive  in  establishing 
meaningful  O&S  eost  constraints  and  actually  controlling  cost  growth;  however, 

-  to  properly  inform  aequisition  deeision  makers,  future  portfolio  O&S  eosts  should 
be  dehned  so  that  the  true  ineremental  eost  of  deploying  a  new  system  is  elearly 
revealed. 

Recommendations 

•  Continuing  top-level  management  dedieation  is  essential  for  proeesses  to  eontrol 
affordability  to  be  effective.  The  reeommendations  of  IDA  Paper  P-4961  for  an 
effeetive  proeess  to  address  aequisition  program  affordability  apply. 

•  A  standard  O&S  cost  model,  implemented  for  eaeh  major  aequisition  program  at 
Milestone  A,  would  greatly  faeilitate  doeumenting  framing  assumptions,  tracking  O&S 
cost  affordability,  and  facilitating  O&S  cost  tradeoffs. 

•  Continued  stress  on  O&S  costs  and  affordability  at  both  Defense  Aequisition  Board  and 
Defense  Aequisition  Exeeutive  Summary  reviews  will  more  strongly  motivate 
Government  managers  and  their  industry  eounterparts  to  eontrol  O&S  eosts. 

-  Future  poliey  issuanees  should  elarify  that  assessing  the  affordability  of  O&S  eosts 
should  be  made  using  the  portfolio  of  forees  that  will  be  altered  with  deployment  of 
the  new  system. 

-  If  future  portfolio  foree  O&S  eosts  will  inerease  with  the  fielding  of  the  proposed 
system,  the  decision  maker  should  insist  that  the  sponsoring  eomponent  provide  a 
realistic  plan  for  how  the  inerease  will  be  paid  for  under  approved  assumptions 
regarding  future  hscal  resources  expeeted  to  be  available  to  the  eomponent. 
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1.  Introduction 


This  research  effort  was  sponsored  by  the  Director,  Acquisition  Resources  and  Analysis,  in 
the  Office  of  the  Under  Secretary  of  Defense,  Acquisition,  Technology  and  Logistics 
(OUSD(AT&L)  and  builds  upon  previous  research  in  IDA  Paper  P-4961,  Affordability  of 
Defense  Acquisition  Programs,  by  the  Institute  for  Defense  Analyses  (IDA)/  The  first  two 
chapters  of  P-4691  are  equally  applicable  to  this  paper,  and  will  not  be  repeated  here.  P-4961 
focused  on  the  overall  context  and  theory  of  assessing  affordability.  This  paper  has  a  more 
specific  focus  on  two  areas:  (1)  what  can  be  learned  from  historical  efforts  to  control  the 
affordability  of  Department  of  Defense  (DOD)  programs,  and  (2)  how  can  affordability 
constraints  for  sustainment  costs,  also  called  operating  and  support  costs,  best  be  implemented. 

The  Better  Buying  Power  (BBP)  initiatives  have  as  their  first  provision  the  need  to  “target 
affordability  and  control  cost  growth,”  more  specifically,  mandating  “affordability  as  a 
requirement”  by  setting  affordability  targets  as  key  performance  parameters  (KPPs)  at  Defense 
Acquisition  Milestone  A  (entry  into  the  Technology  Maturation  and  Risk  Reduction  (TMRR) 
phase  of  a  new  acquisition  program)  and  requiring  engineering  trade  studies  that  show  how  each 
key  design  feature  affects  costs  at  Defense  Acquisition  Milestone  B  (entry  into  the  Engineering 
and  Manufacturing  Development  (EMD)  phase).  (The  BBP  was  institutionalized  with  the 
promulgation  of  a  new  Department  of  Defense  Instruction  (DODI)  5000.02  in  November  2013.) 

BBP  is  not  the  first  attempt  by  Defense  Acquisition  Executives  (DAE)  to  promulgate 
policies  to  ensure  that  DOD  acquisition  programs  will  be  affordable.  Two  past  efforts  are 
particularly  noteworthy,  the  Design  to  Cost  (DTC)  initiative  that  began  in  the  late  1970s,  and  the 
Cost  as  an  Independent  Variable  (CAIV)  concept  of  the  late  1990s.  Although  neither  DTC  nor 
CAIV  were  exactly  the  same  as  the  BBP  affordability  initiatives,  there  is  sufficient  similarity  to 
expect  that  useful  lessons  can  be  learned  from  examining  the  Department’s  experience  with  the 
two  programs.  This  premise  motivated  the  first  part  of  this  paper.  The  similarities  and  differences 
will  be  explored  in  Chapter  2. 

Even  though  sustainment  or  operating  and  support  (O&S)  costs  for  an  acquisition  program 
typically  comprise  60-70  percent  of  a  program’s  total  life-cycle  cost,  these  costs  have  received 


Gene  H.  Porter  et  al.,  Affordability  of  Defense  Acquisition  Programs,  IDA  Paper  P-4961  (Alexandria,  VA: 
Institute  for  Defense  Analyses  (IDA),  April  2013). 

2 

Ashton  B.  Carter,  Memorandum  for  Acquisition  Professionals,  “Subject:  “Better  Buying  Power:  Guidance  for 
Obtaining  Greater  Efficiency  and  Productivity  in  Defense  Spending”  (Washington,  DC:  Department  of  Defense 
(DOD),  September  14,  2010),  2. 

^  Interim  DOD  Instruction  (DODI)  5000.02,  “Operation  of  the  Defense  Acquisition  System,”  November  25,  2013. 
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less  attention  in  the  past,  though  less  does  not  mean  none.  In  faet,  both  DTC  and  CAIV  in  their 
poliey  formulations  ineluded  O&S  eosts.  In  praetiee,  however,  the  IDA  researeh  team  found  little 
evidenee  that  control  of  O&S  costs  achieved  much  success  in  either  the  DTC  or  CAIV  efforts. 
There  are,  however,  reasons  that  support  the  hope  that  this  time  is  different  because  there  is  good 
evidence  that  the  BBP  initiatives  regarding  O&S  affordability  have  been  institutionalized  to  a 
considerable  extent.  That  said,  this  institutionalization  is  still  too  recent  to  make  judgments  about 
its  effectiveness  in  controlling  O&S  costs  for  new  acquisition  programs. 

This  research  draws  on  other  past  IDA  work  regarding  the  DOD  acquisition  process.  In 
2009  IDA  conducted  in-depth  research  on  cost  growth  in  eleven  Major  Defense  Acquisition 
Programs  (MDAP)  between  1999  and  2009.  Since  several  of  those  programs  advertised 
themselves  as  employing  the  CAIV  concept,  the  IDA  team  was  able  to  draw  on  that  prior  work 
to  gain  insights  into  the  success  of  CAIV.  It  is  also  fortunate  that  IDA  performed  analyses  of  the 
effectiveness  of  acquisition  reform  initiatives  of  the  1980s  era  that  included  DTC. 

IDA  has  also  conducted  considerable  research  on  the  O&S  costs  of  weapon  systems  that 
informed  this  paper’s  investigation  into  affordability  of  sustainment  costs;  however,  that  research 
does  not  include  actual  assessments  of  the  success  in  controlling  O&S  costs  within  a  cost  cap,  as 
was  the  case  with  DTC.  The  research  team’s  investigation  of  O&S  cost  affordability  did  not  have 
an  equivalent  base  of  IDA  work  to  build  upon.  The  two  primary  sources  used  were  O&S  cost 
estimates  reported  in  Selected  Acquisition  Reports  (SAR)  and  data  from  the  DOD  Visibility  & 
Management  of  Operating  &  Support  Cost  (VAMOSC)  systems,  which  report  actual  O&S  costs 
for  certain  DOD  systems,  with  varying  levels  of  completeness  and  fidelity.  The  SARs  provide 
O&S  cost  estimates  during  the  acquisition  phases  of  a  system  while  VAMOCS  reports  costs  after 
the  system  is  fielded.  At  some  point  after  initial  fielding,  SAR  O&S  costs  should  reflect 
VAMOSC  experience,  and  the  research  team  found  that  usually  was  the  case  for  the  limited 
number  of  systems  for  which  O&S  costs  were  analyzed.  A  systematic  comparison  of  SAR  O&S 
cost  estimates  to  VAMOSC  data  was  beyond  the  scope  of  this  research.  Available  data  did  allow 
the  research  team  to  provide  some  useful  insights  regarding  the  costs  of  a  new  system  relative  to 
the  system  or  systems  it  will  replace  (“antecedent  systems”)  and  into  the  changes  over  time  of 
SAR  O&S  cost  estimates  as  programs  develop. 

Lastly  the  research  team  investigated  how  O&S  costs  might  be  better  tracked  and  how 
programs  might  be  adjusted  to  achieve  more  affordable  fielded  systems  and  increase  the 
motivation  of  both  government  and  industry  managers  to  give  more  weight  to  O&S  costs  in 
systems  development. 
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2.  History  of  Constraining  Costs  in  Acquisition 

Programs 


The  documentary  record  of  attempts  to  constrain  the  costs  of  large-scale  arms  development 
programs  to  some  pre-specified  level  extends  back  at  least  four  centuries,  and  there  are  even 
suggestions  that  the  classical  civilizations  of  ancient  Greece  and  ancient  China  pursued  such 
efforts.  There  are,  however,  few  records  of  success  in  any  period."^  The  modern  history  of  cost- 
constrained  programs  in  DOD  begins  with  Total  Package  Procurement  (TPP)  in  the  1960s. 
Under  TPP  the  prime  contractor  took  contractual  responsibility  for  delivering  the  system  at  an 
essentially  fixed  life  cycle  cost  (LCC).  Almost  without  exception,  TPP  programs  experienced 
unusually  high  cost  growth  and  encountered  a  variety  of  other  ills,  giving  the  entire  concept  a 
bad  name. 

A.  Better  Buying  Power  (BBP)  versus  Design  to  Cost  (DTC)  and  Cost  as  an 
Independent  Variable  (CAIV) 

BBP  was  institutionalized  in  the  new  DOD  Instruction  5000.02,  published  as  an  interim 
document  while  this  analysis  was  under  way  in  November,  2013.  As  the  best  representation  of 
the  affordability  provisions  of  BBP,  DODI  5000.02  will  be  used  to  develop  comparisons 
between  BBP,  DTC,  and  CAIV.  To  do  so,  it  is  useful  to  review  the  more  salient  sections  of  the 
instruction  regarding  affordability. 

1.  The  Major  Provisions  of  BBP  Regarding  Affordability 

In  accordance  with  BBP  2.0,  the  new  5000.02  specifies  the  establishment  of  “affordability 
goals  and  caps”  for  all  acquisition  programs.^  DODI  5000.02  does  not  require  a  formal 
affordability  assessment  for  the  Materiel  Development  Decision  (MDD),  which  is  the  entry  point 
of  a  new  program  into  the  DOD  acquisition  process.  However,  Enclosure  8,  “Affordability 
Analysis  and  Investment  Constraints,”  suggests 

...some  [affordability]  analysis  even  before  that  point  [i.e.,  the  MDD]  is 
beneficial.  The  best  opportunity  for  ensuring  that  a  program  will  be  affordable  is 
through  requirements  tailoring  that  occurs  before  and  during  the  AoA.  Thus,  the 
Components  will  incorporate  estimated  funding  streams  for  future  programs 
within  their  affordability  analyses  at  the  earliest  conceptual  point  and  specify 


Carla  Rahn  Phillips,  Six  Galleons  for  the  King  of  Spain:  Imperial  Defense  in  the  Early  Seventeenth  Century 
(Baltimore,  MD:  Johns  Hopkins  University  Press,  1996). 

Defined  by  the  new  5000.02  as  follows:  “Affordability  constraints  are  established  to  inform  the  Program 
Manager  and  the  requirements  authority  of  the  cost  limitations  dictated  by  the  Component’s  affordability 
analysis.  Affordability  goals  are  key  objectives  set  to  inform  requirements  and  design  tradeoffs  during  early 
research  and  development.  Affordability  caps  are  fixed  requirements  equivalent  to  Key  Performance  Parameters 
(KPPs).” 
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those  estimates  at  MDD  and  earlier  to  inform  system  design  eoneepts  and 
alternative  seleetion.^ 

However,  during  the  Materiel  Solutions  Phase,  between  the  MDD  and  Milestone  A  (entry 
into  TMRR),  a  more  eomplete  and  formal  affordability  assessment  is  required  to  support  the 
establishment  of  an  affordability  goal  for  the  program  at  Milestone  A.  Such  an  assessment  is  also 
needed  to  support  the  Analysis  of  Alternatives  (AoA),  which  is  the  major  activity  conducted  in 
the  Materiel  Solutions  Phase,  to  ensure  that  the  alternative  systems  costs  (both  investment  and 
O&S)  considered  in  the  AoA’s  cost-effectiveness  comparisons  are  realistic.^ 

Based  on  analysis  provided  by  the  sponsoring  DOD  Component  at  Milestone  A,  the 
Milestone  Decision  Authority  (MDA)  approves  affordability  goals  for  both  unit  procurement  and 
sustainment  (but  not  for  research,  development,  test  and  evaluation  (RDT&E)),  to  be 
documented  in  the  Acquisition  Decision  Memorandum  (ADM).  Specifically,  the  Component 
must 

Present  an  Affordability  Analysis  and  proposed  goals  based  on  the  resources  that 
are  projected  to  be  available  in  the  portfolio(s)  or  mission  area(s)  associated  with 
the  program  under  consideration.  The  analysis  will  be  supported  by  a  quantitative 
assessment  of  all  of  the  programs  in  the  prospective  program’s  portfolio/mission 
area  that  demonstrates  the  ability  of  the  Component’s  estimated  budgets  to  fund 
the  new  program  over  its  planned  life  cycle.  Affordability  Analyses  do  not 
produce  rigid,  long-range  plans;  their  purpose  is  to  inform  current  decisions  about 
the  reasonableness  of  embarking  on  long-term  capital  investments  at  specific 
capability  levels.  The  Affordability  Analysis  will  support  the  Component’s 
proposed  affordability  goals  for  unit  production  and  sustainment  costs  for  MDA 
approval  and  recording  in  the  Milestone  A  ADM.^ 

During  the  TMRR  phase  (between  Milestones  A  and  B),  the  affordability  goal  is  used  in 
requirements  and  systems  engineering  trade  studies  to  ensure  that  the  program  will  ultimately  be 
affordable.  Sustainment  affordability  is  included: 

As  the  product  support  package  design  matures,  the  Program  Manager  will  ensure 
that  life-cycle  affordability  is  a  factor  in  engineering  and  sustainment  trades.^ 

At  Milestone  B,  an  updated  affordability  assessment  is  required  and  is  used  by  the  MDA  to 
specify  the  program’s  affordability  caps  for  unit  procurement  and  sustainment  costs.  These  caps 
are  intended  to  be  equal  in  importance  to  performance  requirements  as  the  program  proceeds 
through  EMD,  and  they  are  to  be  recorded  in  the  Acquisition  Program  Baseline  (APB).  An 


^  DODI  5000.02,  120. 

^  “The  analysis  of  alternatives  will  inform  and  be  informed  by  affordability,  sustainability,  early  systems 
engineering  analyses,  and  industry  studies.”  DODI  5000.02,  16. 

g 

Ibid.,  16.  The  portfolio  aspects  of  this  proviso  will  be  discussed  subsequently. 

Ibid.,  20. 
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additional  update  is  required  before  Milestone  C,  entry  into  the  Production  and  Deployment 
Phase. 


As  noted  above,  Enclosure  8  of  the  new  DODI  5000.02  is  entitled  “Affordability  Analysis 
and  Investment  Constraints.”  The  five-page  document  explains  the  requirements  for  affordability 
analyses  needed  at  Milestone  reviews  in  considerable  detail. 

A  key  point  of  the  document  is  that  affordability  must  be  determined  by  the  DOD 
Components  in  the  context  of  a  set  of  portfolios.  The  enclosure  states  that 

If  they  are  provided,  Components  will  use  office  of  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics  standardized  portfolios  for 
their  analysis.  Portfolios  can  be  based  on  mission  areas  or  commodity  types,  and 
will  define  a  collection  of  products  that  can  be  managed  together  for  investment 
analysis  and  oversight  purposes.  Components  will  normally  make  tradeoffs  within 
portfolios,  but  if  necessary,  can  and  should  make  tradeoffs  across  portfolios  to 
provide  adequate  resources  for  high-priority  programs. 

Ideally,  the  Components’  portfolios  will  add  up  to  its  projection  of  total  obligational 
authority  (TOA)  expected  to  be  available  under  realistic  fiscal  constraints.  (The  enclosure  states: 
“The  aggregation  of  portfolio  cost  estimates  for  each  year,  when  combined  with  all  other  fiscal 
demands  on  the  Component,  may  not  exceed  the  Component’s  reasonably  anticipated  future 
budget  levels. ’’(page  119))  Frequently,  however.  Components  tend  to  consider  only  investment 
portfolios  when  viewing  the  affordability  of  program  investment  costs  (consistent  with  the 
definition  of  portfolios  by  “commodity  types”  cited  in  the  paragraph  above).  Sustainment  costs 
require  a  different  portfolio  view,  which  can  be  problematic,  since  it  is  sometimes  difficult  to 
draw  plausible  portfolio  boundaries  for  O&S  costs.  Enclosure  8  is  somewhat  vague  on  the 
requirement  to  submit  data  supporting  its  affordability  assessments,  stating  that 

Each  Component’s  affordability  analysis  is  presented  within  the  governance 
framework  to  the  MDA  in  preparation  for  major  acquisition  decisions  in  a  format 
that  demonstrates  the  affordability  of  the  program  within  the  Component  and 
portfolio  context,  to  ensure  that  the  resulting  affordability  constraints  are 
understood  and  consistent  with  the  future  total  budget  projection. 

And: 

...In  general,  standardized  stacked  area  charts  (or  “sand  charts”)  and  spreadsheets 
listing  the  estimated  budget  by  year  for  each  element  of  the  analysis,  are  adequate. 

The  data  should  compare  life-cycle  estimates  to  the  historical  experience  within 
the  portfolio  and  the  Component  for  sustainment  and  procurement  costs. 

...At  each  major  acquisition  decision  meeting,  the  DOD  Component  will  provide 
stacked  area  charts  (“sand  charts”)  and  underlying  spreadsheet  data  showing  the 
program’s  budget,  what  portfolio  it  fits  within,  and  the  top-level  total  of  all 
portfolios  and  accounts  totaling  at  or  below  the  future  total  budget  projection. 


Ibid.,  118. 
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equivalent  to  Total  Obligation  Authority.  Additional  detail  and  samples  of  the 
sand  eharts  are  presented  in  the  Defense  Aequisition  Guidebook.  ^  ^ 

These  provisions  do  not  appear  to  require  the  submissions  to  include  the  full  programmatic 
contents  of  the  portfolios.  The  objective  is,  thus,  to  specify  affordability  goals  and  caps  so  that  all 
the  programs  in  a  portfolio  are  executable  within  the  postulated  fiscal  constraints  on  the 
portfolio,  on  the  Component,  and  on  the  Department. 

Since  affordability  constraints  are  to  be  determined  based  on  the  fiscal  resources  available 
in  the  appropriate  portfolio,  the  new  DODI  5000.02  argues  that  affordability  goals  and  caps  are 
distinct  from  programs’  approved  cost  estimates: 

Affordability  analysis  and  affordability  constraints  are  not  synonymous  with  cost 
estimation  and  approaches  for  reducing  costs.  Constraints  are  determined  in  a  top- 
down  manner  by  the  resources  a  Component  can  allocate  for  a  system,  given 
inventory  objectives  and  all  other  fiscal  demands  on  the  Component.  Constraints 
then  provide  a  threshold  for  procurement  and  sustainment  costs  that  cannot  be 
exceeded  by  the  Program  Manager.  On  the  other  hand,  cost  estimates  are 
generated  in  a  bottom-up  or  parametric  manner  and  provide  a  forecast  of  what  a 
product  will  cost  for  budgeting  purposes.  The  difference  between  the  affordability 
constraints  and  the  cost  estimates  indicates  whether  actions  must  be  taken  to 
further  reduce  cost  in  order  to  remain  within  affordability  constraints. 
Independent  of  affordability  constraints  or  cost  estimates,  program  managers 
should  always  be  looking  for  ways  to  control  or  reduce  cost.  Proactive  cost 
control  is  central  to  maximizing  the  buying  power  of  the  Department  and  should 

be  an  integral  part  of  all  phases  and  aspects  of  program  management.  Cost  control 

12 

approaches  are  discussed  in  Enclosure  10  of  this  instruction. 

This  point  is  worthy  of  discussion:  Clearly  there  must  be  a  close  relationship  between 
affordability  constraints  and  program  cost  estimates.  Even  in  the  earliest  stages  of  program 
initiation,  there  will  be  some  notion  of  how  much  proposed  system  candidates  will  cost — usually 
within  a  fairly  broad  range.  Such  cost  estimates  would  be  taken  into  consideration  in  determining 
whether  the  projected  funds  for  the  portfolio  are  sufficient  for  the  proposed  new  program.  If, 
after  making  whatever  portfolio  adjustments  might  be  required  (either  in  program  content  or 
available  resources),  it  appears  to  Component  planners  that  an  acquisition  new  start  can  be 
proposed,  then  the  initial  affordability  goal  is  implied  by  the  funds  allocated  to  it  within  the 
portfolio.  As  program  plans  advance,  the  cost  estimates  will  be  refined,  the  range  of  uncertainty 
reduced,  and  the  portfolio  projection  adjusted  for  consistency. 

DODI  5000.02  also  addresses  the  important  issue  of  enforcement  of  affordability 
constraints: 


“  Ibid.,  120. 
Ibid.,  117. 
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When  approved  affordability  eonstraints  eannot  be  met — even  with  aggressive 
cost  control  and  reduction  approaches — then  technical  requirements,  schedule, 
and  required  quantities  must  be  revisited;  this  will  be  accomplished  with  support 
from  the  Component’s  Configuration  Steering  Board,  and  with  any  requirements 
reductions  proposed  to  the  validation  authority.  If  constraints  still  cannot  be  met, 
and  the  Component  cannot  afford  to  raise  the  program ’s  affordability  cap(s)  by 
lowering  constraints  elsewhere  and  obtaining  MDA  approval,  then  the  program 
will  be  cancelled.  (emphasis  added) 

The  sentence  in  italics  in  the  quotation  above  deserves  elaboration.  It  will  be  necessary  to 
periodically  review  the  portfolio  and  its  projected  funding  to  ensure  that  the  previously- 
determined  tentative  goals  are  still  viable,  and  if  not,  appropriate  adjustments  made — either  to 
program  content  and/or  timing,  to  other  programs  in  the  portfolio,  or  even  to  the  funds  that  can 
be  made  available  for  the  portfolio.  This  is  an  ongoing  process  as  the  program  advances  (or  not), 
eventually  passing  Milestone  B,  when  the  affordability  goals  become  firmer  caps.  At  that  point 
the  program’s  cost  estimates  will  probably  link  closely  to  the  affordability  caps. 

If  at  any  point  in  the  acquisition  process  the  projected  costs  of  a  program  exceed  the 
affordability  constraint  that  has  been  established,  a  multi-prong  process  should  be  initiated.  For 
example,  in  case  A:  The  program  is  healthy  but  the  costs  estimates  on  which  the  allocation  of 
portfolio  funds  to  the  program  were  based  on  assumptions  now  recognized  as  invalid.  A  better- 
informed  cost  estimate  is  available.  The  issue  is  thus  whether  the  program  still  promises 
sufficient  capability  benefits  to  justify  its  new  cost  and  whether  sufficient  offsetting  funds  can  be 
made  available,  either  within  the  portfolio  or  external  to  it.  If  so,  then  the  program  should 
proceed  under  revised  affordability  constraints.  If  not,  cancellation  is  the  alternative.  Case  B;  The 
program  is  not  healthy,  and  costs  have  gone  out  of  control.  On  its  current  path,  it  will  not  be 
affordable.  In  this  case,  it  is  a  question  of  whether  the  program  can  be  put  back  on  track  with 
reasonable  certainty.  Probably  requirements  will  need  to  be  changed,  and/or  less  risky 
technologies  employed.  If  successful,  then  the  restructured  program  might  proceed.  It  may  also 
become  apparent  that  the  program’s  initial  cost  estimates  were  faulty,  which  puts  the  decision 
back  to  Case  A.  Still  other  complicating  conditions  can  arise,  such  as  a  change  in  the 
Department’s  or  the  Component’s  fiscal  projection  (as  has  recently  occurred).  In  that  case,  the 
funds  originally  projected  for  the  portfolio  may  no  longer  be  realistic,  so  the  original 
affordability  constraints  are  no  longer  applicable.  Or  there  might  be  a  substantial  change  in  the 
strategic  environment  (such  as  after  the  terrorist  attacks  of  September  11,  2001  (9/11)),  which 
implies  major  shifts  in  priorities.  Either  of  those  situations  might  mean  that  a  perfectly  healthy 
program  proceeding  within  its  affordability  constraints  may  have  to  be  cancelled  or  delayed  in 
favor  of  now  higher  priority  programs.  The  point  of  this  discussion  is  that  program  cancellation 
is  only  one  possible  outcome  of  a  situation  in  which,  for  diverse  reasons,  a  program  cannot 
remain  within  its  affordability  constraints,  which  themselves  cannot  be  considered  sacrosanct. 


Ibid.,  118. 
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Affordability  caps  are  to  be  enforced  by  the  MDA;  and  program  managers  (PM)  that  begin 
to  question  whether  their  program  can  remain  within  affordability  eaps  are  to  promptly  report  the 
status  to  the  MDA.  Enclosure  8  states  that  affordability  should  be  also  a  topic  for  review  under 
the  Defense  Aequisition  Exeeutive  Summary  (DAES)  process. 

Another  point  worth  discussion  before  proeeeding  with  the  comparisons  between  BBP, 
DEC  and  CAIV  coneerns  the  should-cost  provisions  of  BBP.  Should-eost  is  defined  in  DODI 
5000.02  as  follows: 

‘Should  Cost’  is  a  management  tool  designed  to  proaetively  target  cost  reduction 
and  drive  productivity  improvement  into  programs.  Should  cost  management 
challenges  managers  to  identify  and  achieve  savings  below  budgeted  most  likely 
costs. 

Should-cost  goals  are  thus  well  below  affordability  constraints  and  serve  very  different 
operational  ends.  A  program  would  not  be  penalized  for  failing  to  meet  a  should-eost  goal, 
whereas  that  would  elearly  not  be  the  case  of  an  affordability  eap.  As  will  be  seen,  DEC  and 
CAfV  both  had  a  should-cost  flavor  in  addition  to  their  relationships  to  affordability.  In  contrast, 
BBP  makes  a  elear  distinetion  between  aggressive  should-eost  goals,  and  more  conservative  and 
realistic  program  baselines.  Not  making  that  distinetion  in  the  DEC  and  CAIV  implementations 
might,  in  fact,  be  a  key  reason  for  their  failure.  However,  there  are  differences  between  should- 
eost  goals  today  and  DEC  and  CAIV  target  costs.  Eoday’s  should-cost  goals  are  not  expeeted  to 
be  aehieved  without  undertaking  some  exceptional  actions  to  reduce  eost  below  budgeted  levels. 
Eurthermore,  it  is  understood  that  such  actions  may  not  achieve  the  desired  results— there  are 
risks  involved. 

B.  Design  to  Cost 

1.  History  and  Background 

Although  controlling  costs  has  always  been  an  important  aspect  of  any  development  or 
acquisition  program,  the  first  reference  the  researeh  team  found  to  the  DEC  idea  per  se  was  in  a 
speeeh  by  Director,  Defense  Research  and  Engineering  (DDR&E)  John  Eoster  in  Mareh  1970.  In 
it,  Eoster  said  that  “. .  .priee  has  as  mueh  priority  as  performanee. .  .we  must  design-to-a-priee. .  .or 

1  -j 

else  we  will  not  be  able  to  afford  what  we  need.” 


DAES  is  a  quarterly  process  for  periodic  and  systematic  reviews  of  MDAPs.  Roughly  one-fourth  of  the  MDAPs 
will  be  reviewed  each  quarter  in  a  meeting  with  the  MDA  or  his/her  designate. 

DODI  5000.02,  77. 

See  the  Defense  Acquisition  Guidebook  for  the  discussion  of  should-eost  in 

https://acc.dau. mi l/CommunitvBrowser.aspx?id=5 187 10.  “Unlike  affordability  requirements,  we  do  not  expect 
them  [should-eost  goals]  to  always  be  achieved,  but  we  do  expect  strong  efforts  to  do  so”  (section  10.15.2). 

See  Joseph  R.  Busek,  “A  Historical  Analysis  of  Total  Package  Procurement,  Life  Cycle  Costing,  and  Design  to 
Cost,”  AD-A030-141,  U.S.  Department  of  Commerce,  National  Technical  Information  Service,  June  1976.  This 
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In  1971  the  first  DOD  Directive  on  major  systems  acquisition  was  issued  by  Deputy 
Secretary  David  Packard.  This  document,  only  six  pages  in  length,  is  an  excellent  description 
of  the  necessary  characteristics  of  the  defense  acquisition  process,  and  is  as  applicable  today  as  it 
was  in  1971.  It  makes  explicit  mention  of  design  to  cost: 

Cost  parameters  shall  be  established  which  consider  the  cost  of  acquisition  and 
ownership;  discrete  cost  elements  (e.g.,  unit  production  cost,  operating  and 
support  cost)  shall  be  translated  into  “design-to”  requirements.  System 
development  shall  be  continuously  evaluated  against  these  requirements  with  the 
same  rigor  as  that  applied  to  technical  requirements.  Practical  tradeoffs  shall  be 
made  between  system  capability,  cost  and  schedule.  Traceability  of  estimates  and 
costing  factors,  including  those  for  economic  escalation,  shall  be  maintained. 
(Section  III.C.2) 

The  1975  reissue  of  DODD  5000.1  contained  the  same  provision  and  DODD  5000.28, 
dedicated  to  DTC,  was  promulgated  the  same  year.  In  it  DTC  is  defined  as: 

A  management  concept  wherein  rigorous  cost  goals  are  established  during 
development  and  the  control  of  systems  costs  (acquisition,  operating  and  support) 
to  these  goals  is  achieved  by  practical  tradeoffs  between  operational  capability, 
performance,  cost,  and  schedule.  Cost,  as  a  key  design  parameter,  is  addressed  on 

a  continuing  basis  and  as  an  inherent  part  of  the  development  and  production 

20 

process. 

While  from  a  program  management  standpoint,  a  DTC  goal  is  much  like  an  affordability 
constraint,  the  directive  does  not  clearly  state  the  relationship  between  establishing  the  goal  and 
an  assessment  of  funds  likely  to  be  available.  Regarding  establishing  the  DTC  goal,  DODD 
5000.28  stated  that 

An  initial  estimate  of  the  resources  available  for  allocation  to  the  program  shall  be 
made  and  cost  objectives  established  during  concept  formulation.  Likewise,  the 
minimum  essential  performance  characteristics  shall  be  quantified  to  avoid 
tradeoffs  below  that  necessary  to  satisfy  the  required  operational  capability.  Each 
technically  feasible  alternative  will  be  analyzed  and  cost/performance  tradeoffs 


is  a  master’s  thesis  at  the  Air  Force  Institute  of  Technology  that  provides  references  to  a  number  of  other 
speeches  and  memoranda  from  the  formative  period  of  DTC. 

DODD  5000.1,  July  13,  1971.  This  is  the  first  version  that  appears  in  the  Pentagon  Digital  Library  at 
http://www.whs.mil/librarv/mildoc/DODD.html.  See  also  Joe  Ferrara,  “DOD’s  5000  Documents:  Evolution  and 
Change  in  Defense  Acquisition  Policy,”  Acquisition  Review  Quarterly  (Fall  1996),  which  states  that  the  July 
1971  issuance  is  the  first. 

Regarding  TPP,  the  document  stipulates:  “It  is  not  possible  to  determine  the  precise  production  cost  of  a  new 
complex  defense  system  before  it  is  developed;  therefore,  such  systems  will  not  be  procured  using  the  total 
package  procurement  concept  or  production  options  that  are  contractually  priced  in  the  development  contracf  ’ 
(Section  III.C.7). 

DODD  5000.28,  May  23,  1975,  “Design  to  Cost.”  This  directive  cancels  two  previous  Deputy  Secretary 
memoranda  that  addressed  DTC,  one  from  June  1973  and  one  from  May  1974.  These  memos  have  not  been 
located. 
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made  to  ensure  seleetion  of  the  lowest  life  cycle  cost  solution.  As  soon  as  the 
system  is  definitized  to  the  extent  that  cost  associated  with  minimum  performance 
needed  can  be  estimated  with  confidence,  a  firm  Design  to  Cost  goal  shall  be 
recommended  for  the  program. 

However,  the  directive  gives  no  guidance  on  how  to  make  that  initial  resource  estimate.  A  more 
detailed  implementation  guide  provided  somewhat  greater  specificity  regarding  the  steps  needed 
to  establish  DTC  goals: 

Every  system  has  many  parameters  which  must  be  considered  in  design.  ... 
Because  of  this  multiplicity  of  considerations,  there  are  a  great  number  of  possible 
combinations  of  values  for  each.  At  the  outset  of  an  acquisition,  the  optimum 
combination  cannot  be  identified.  However,  certain  limits  must  be  identified. 

Given  a  threat  environment,  level  of  technology  and  mission  scenario,  there  are 
dictated  certain  minimum  essential  performance  parameters,  the  values  of  which 
must  be  attained  or  the  system  is  not  mission  capable.  There  also  results  a  certain 
cost  which  must  be  achieved  or  bettered  or  the  system  is  not  affordable.  The 
solution  can  be  visualized  as  follows: 

COST  CEILING 

RANGE  OE  ACCEPTABLE  SOLUTIONS 
PEREORMANCE  ELOOR 

These  limits  fix  the  area  in  which  the  optimum  combination  of  performance,  cost, 
and  schedule  values  must  fall.  In  this  area,  it  may  be  found  that  performance  and 
schedule  values  above  the  minimum  established  requirement  are  useful  and  can 
be  obtained  within  the  affordable  cost.  Performance  and  schedule  values  above 

minimum  will  also  be  found  for  which  the  added  cost,  although  within  the  limits 

22 

of  affordability,  are  unjustifiable  given  the  utility  of  the  added  performance. 

Again,  this  falls  well  short  of  the  specificity  contained  in  the  Affordability  Enclosure  of  the 
current  new  DODI  5000.02.  Nonetheless,  these  documents  are  remarkably  close  in  many 
respects  to  the  affordability  provisions  of  BBP,  with  the  suggestion  in  some  of  the  descriptions  of 
a  similarity  with  should-cost. 

The  difference  between  current  affordability  constraints  and  DTC  targets  is  clearly 
illustrated  by  the  quote  below  from  DODD  5000.28: 

The  recommended  Design  to  Cost  goal  should  be  a  difficult  but  achievable 
objective  which  should  challenge  designers,  engineers,  and  program  managers  to 
their  best  efforts.  Care  must  be  exercised  to  ensure  the  goal  is  properly  selected;  a 


DODD  5000.28,  4. 

Department  of  the  Army,  the  Navy  and  the  Air  Force,  Joint  Design-to-Cost  Guide:  Life  Cycle  Cost  as  a  Design 
Parameter  (Alexandria,  VA:  Army  Materiel  Development  and  Readiness  Command,  15  October  1977)  (a  re¬ 
issue  of  an  earlier  1975  document  which  has  not  been  found).  This  document  was  published  under  the  signature 
of  the  Army,  Navy,  and  Air  Force  systems  acquisition/  materiel  commanders  (and  in  the  case  of  the  Air  Force, 
the  logistics  commander  in  addition),  3. 
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goal  which  is  too  high  in  relation  to  the  required  performanee  wastes  money  and 
an  exeessively  low  goal  sets  the  stage  for  eost  growth,  buy-ins,  or  unaeoeptable 
systems. 

This  aspect  of  DTC  is  a  conundrum.  If  the  eost  goal  is  set  low  enough  to  be  “difficult,” 
then  it  should  be  expeeted  that  many  programs  will  be  unable  to  achieve  them.  Sueh  programs 
will  require  high-level  management  attention  (Seeretary-level).  The  BBP  approaeh  avoids  that 
problem  by  setting  two  goals — if  set  correctly  the  affordability  eonstraint  should  seldom  be 
violated,  whereas  the  should-cost  goal  can  be  set  more  aggressively  with  a  mueh  higher  risk  of 
non-aehievement  but  lower  eonsequenees.  While  programs  are  expeeted  to  routinely  make  their 
“best  efforts”  to  reduce  costs,  programs  are  expected  to  identify  initiatives  that  are  effective  in 
redueing  eosts,  eonsistent  with  the  risks.  There  is  no  assumption  that  eost  ean  be  redueed  if 
people  perform  their  jobs  well  as  seems  to  have  been  the  philosophy  of  DTC. 

DODD  5000.28  was  eaneelled  by  the  issuanee  of  DODD  4245.3  in  April  1983,  also 
entitled  “Design  to  Cost.”  Though  eoneeptually  the  same,  there  are  some  important  differenees 
and  refinements  that  both  bring  the  original  doeument  up  to  date  in  terms  of  proeess  and  speeify 
more  detail  in  implementation.  One  important  differenee  is  that  DODD  4245.3  assigns  speeifie 
responsibilities  for  formulating  and  traeking  DTC  goals.  The  Direetor  of  Program  Analysis  and 
Evaluation  (PA&E)  is  required  to  “review  DTC  parameters  to  ensure  that  they  are  feasible,  are 
based  on  the  performanee  and  design  speeifieations  required  to  meet  mission  objeetives,  and 
identify  high-eost  or  high-risk  elements  of  the  system.”^"^  The  Direetor  of  the  Cost  Analysis 
Improvement  Group  (CAIG),  is  required  to  “provide  to  the  DAE,  at  eaeh  DSARC  review,  an 
independent  assessment  of  the  reeommended  DTC  goals  and  thresholds. The  Under  Seeretary 
of  Defense  (Comptroller)  (USD(C))  is  required  to  “ensure  that  the  methodology  developed  by 
DOD  Component  heads  for  determining  actual  or  estimated  eosts  is  uniform  and  eonsistently 
applied  and  that  DTC  parameters  are  eompatible  with  resource  planning  documents  sueh  as  the 
Eive-Year  Defense  Plan  and  the  Extended  Planning  Annex.”  There  is  nothing  eomparable  to 
these  provisions  in  the  new  DODI  5000.02.^^ 


DODD  5000.28,  4. 

DODD  4245.3,  5. 

DODD  4245.3,  5-6. 

Those  responsibilities  should  logically  have  been  given  to  Director,  PA&E,  who  had  overall  responsibility  for 
the  FYDP  and  Extended  Planning  Annexes.  And  cost  methodologies  are  logically  the  purview  of  the  CAIG, 
which  was  also  under  Director,  PA&E. 

There  is  a  question  as  to  whether  these  provisions  were  routinely  implemented.  Several  Decision  Coordination 
Papers  (DCP)  of  the  era  were  examined  and  no  DTC  provisions  were  found.  Also,  individuals  who  were  on  the 
DDR&E  staff  in  the  mid/late  1970s  do  not  recall  DTC  goals  being  subjects  of  discussion  with  leadership  nor 
working  any  actions  to  modify  DTC  goals.  This  is  a  subject  that  deserves  more  investigation.  [The  DCP  is 
described  in  the  1971  issuance  of  DODD  5000.1  in  the  following  terms:  “The  considerations  which  support  the 
determination  of  the  need  for  a  system  program,  together  with  a  plan  for  that  program,  will  be  documented  in 
the  DCP.  The  DCP  will  define  program  issues,  including  special  logistics  problems,  program  objectives. 
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Another  important  difference  between  DODD  5000.28  and  DODD  4245.3  is  an  emphasis 

on  carrying  DTC  goals  forward  into  acquisition  contracts,  to  include  provisions  for  providing 

28 

“financial  reward”  to  contractors  for  achieving  DTC  goals  for  both  acquisition  and  O&S  costs. 
Again,  there  is  nothing  of  comparable  specificity  for  affordability  constraints  (or  should-cost)  in 
the  new  DODI  5000.02.^“^ 

The  January  1977  DODD  5000.1  contains  no  provision  regarding  DTC  per  se,  but  it  does 
list  DODD  5000.28  as  a  reference.  Issuances  of  DODD  5000.1  beyond  the  January  1977  contain 
no  explicit  reference  to  either  DTC  or  to  the  separate  DTC  directives,  though  DODD  4245.3 
remained  in  effect.  The  March  1983  DODI  5000.2  does  reference  DODD  5000.28  (cancelled  one 
month  later  with  the  issuance  of  DODD  4245.3).  The  March  1985  DODI  5000.2  still  references 
the  cancelled  DODD  5000.28  but  not  the  DODD  4245.3  then  in  effect;  however,  the  September 
1987  issuance  correctly  lists  DODD  4245.3  as  a  reference.  The  February  1991  issuance  of 
DODD  5000.01  (page  5-1)  explicitly  cancelled  DODD  4245.3,  and  with  its  cancellation  DTC 
appears  to  have  disappeared  from  DOD  directives  and  instructions  regarding  acquisition.  It  can 
be  concluded  from  this  review  that  DTC,  though  still  “on  the  books,”  received  no  real  emphasis 
in  the  latter  part  of  the  1980s. 

2.  Program  Outcomes  under  DTC 

DTC  was  the  official  policy  of  DOD  for  all  MDAPs  for  almost  two  decades.  This  analysis 
has  found  neither  individuals  nor  documents  that  offer  a  clear  explanation  of  the  reasons  for  its 
demise.  Possibly,  it  was  generally  seen  as  not  helpful  in  controlling  costs.  As  will  be  seen 
below,  such  a  conclusion  is  supportable. 


program  plans,  performance  parameters,  areas  of  major  risk,  system  alternatives  and  acquisition  strategy.” 
There  is  no  modem  counterpart.] 

See  page  6  of  DODD  5000.28.  DODD  4245  states:  “Consideration  shall  be  given  to  making  payment  of  DTC 
incentives  contingent  upon  attaining  required  levels  of  reliability  and  supportability”  (Section  E.4.a(2),  page  4). 

Page  19  of  DODI  5000.02  contains  a  general  provision  in  the  section  on  Acquisition  Strategy:  “The  business 
strategy  will  describe  the  rationale  for  the  contracting  approach  and  how  competition  will  be  maintained 
throughout  the  program  life  cycle,  and  detail  how  contract  incentives  will  be  employed  to  support  the 
Department’s  goals.”  This,  presumably,  would  be  applicable  to  affordability  constraints. 

This  is  confirmed  by  a  remark  in  a  1992  IDA  paper  from  the  period  that  will  be  drawn  upon  extensively  below: 
“While  there  is  still  a  place  in  the  SAR  for  the  design-to-cost  goal,  it  is  often  filled  by  an  ‘N/A.  ’  Of  the  64 
programs  with  FSD  [full-scale  development]  start  in  the  1980s,  only  27  are  showing  a  DTC  goal.”  Karen  W. 
Tyson  et  al..  Acquiring  Major  Systems:  Cost  and  Schedule  Trends  and  Acquisition  Initiative  Effectiveness,  IDA 
Paper  P-2722  (Alexandria,  VA:  IDA,  November  1992),  VIII-2. 

The  early  1980s  surge  in  defense  may  have  translated  into  reduced  urgency  in  controlling  costs.  Such  an 
hypothesis  is  plausible  but  unproven. 

John  Betti  was  Under  Secretary  for  Acquisition  until  December  13,  1990.  Donald  Yockey  was  acting  on  the 
date  of  signature.  Since  DOD  directives  are  drafted  long  before  they  are  signed,  it  is  most  likely  that  the 
decision  to  drop  DTC  was  made  under  Betti. 
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In  most  cases  it  is  impossible  to  adequately  evaluate  the  efficacy  of  any  cost  control 
technique  for  a  number  of  years  after  the  program  starts,  at  least  until  returned  eosts  of 
production  begin  to  beeome  available.  Nonetheless,  in  Mareh  1978,  the  General  Aeeounting 
Office  (GAO)  expressed  eoneems  about  the  implementation  of  DTC  based  on  a  review  of  five 
MDAPs  employing  DTC.^"^  Based  on  previous  IDA  researeh,  it  appears  that  GAO’s  eoneerns 
were  warranted. 

It  is  fortunate  for  this  analysis  that  two  IDA  papers  between  1989  andl992  examined  the 
effeetiveness  of  several  aequisition  reform  eoncepts  instituted  in  the  1980s,  ineluding  DTC.  ’ 
The  eurrent  researeh  has  drawn  on  these  papers  extensively  for  both  insights  and  supporting  data. 
Overall,  the  first  IDA  paper.  The  Effects  of  Management  Initiatives  on  the  Costs  and  Schedules 
of  Defense  Acquisition  Programs,  eoneluded  that  DTC  prineiples  had  been  applied  ineonsistently 
and  had  produeed  no  elear  benefit  in  eontrolling  eost  growth.  Produetion  eost  growth  for  the 
programs  using  DTC  was  on  average  29  pereentage  points  higher  than  that  of  the  non-DTC 
programs  (76  pereent  for  thirty-four  programs  using  DTC  and  47  percent  for  forty-six  programs 
that  did  not).  The  population  of  programs  examined  spanned  from  the  late  1960s  to  the  early 
1980s. 

The  seeond  IDA  paper,  P-2272,  Acquiring  Major  Systems:  Cost  and  Schedule  Trends  and 
Acquisition  Initiative  Effectiveness,  suggested  that  DTC  had  a  positive  effeet  on  production  cost 
growth  on  programs  starting  in  the  late  1970s: 

However,  the  analysis  of  eost  and  sehedule  outeomes  of  programs  with  and 
without  DTC  by  time  period  (Figure  VIII-2)  indicates  that  DTC  programs  starting 
in  the  late  1970s  have  slightly  less  eost  growth  than  non-DTC  programs.  In  this 
time  period  the  eost  growth  of  the  DTC  programs  is  only  5 1  pereent  and  that  of 
the  non-DTC  programs  is  55  pereent  This  may  indieate  that,  by  the  late  1970s,  the 
DTC  eoneept  had  enough  time  to  become  established  and  to  be  applied  early 
enough  in  a  program  to  be  effeetive. 

The  present  analysis  attempted  to  eonfirm  that  hypothesis  through  statistieal  analysis  of 
some  of  the  data  in  Appendix  A  of  IDA  Paper  P-2722.  The  produetion  eost  growths  for 

33 

Since  renamed  the  Government  Accountability  Office. 

GAO,  “The  Department  of  Defense’s  Application  of  the  Design-to-Cost  Concept,”  PSAD-78-79;  B-163058 
(Washington,  DC:  GAO  March  20,  1978).  GAO  concluded  that  DOD  had  not  followed  DTC  principles  closely 
by  not  establishing  cost  targets  at  the  concept  formulation  stage,  by  controlling  near-term  acquisition  costs 
rather  than  life-cycle  costs,  and  by  failing  to  develop  adequate  databases  to  establish  cost-performance 
estimating  relationships.  Only  one  of  the  five  programs  examined  (FFG-7)  appeared  to  have  successfully 
employed  DTC. 

35 

Karen  Tyson  et  al..  The  Effects  of  Management  Initiatives  on  the  Costs  and  Schedules  of  Defense  Acquisition 
Programs,  IDA  Paper  P-2201  (Alexandria,  VA:  IDA,  March  1989);  and  Karen  W.  Tyson  el  al..  Acquiring 
Major  Systems:  Cost  and  Schedule  Trends  and  Acquisition  Initiative  Effectiveness,  IDA  Paper  P-2722 
(Alexandria,  VA:  IDA,  November  1992). 

36 

The  analyses  computed  these  averages  weighted  by  the  programs’  total  cost. 

37 

P-2272  presents  no  statistical  analysis  of  the  data  on  DTC  programs. 
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programs  using  DTC  was  compared  to  that  of  programs  not  using  DTC.  Kolmogorov-Smimov 
tests  were  used  to  compare  the  two  distributions,  and  the  results  are  reported  in  Table  1. 


Table  1.  Results  of  Statistical  Analysis  of  Data  from  1989-92  IDA  Research 


DTC  Programs 

Non-DTC  Programs 

Kolmogorov-SmirnovTest  Results* 

Case 

Number  of 
Programs 

Mean  Production 

Cost  Growth 

Median  Production 

Cost  Growth 

Number  of 
Programs 

Mean  Production 

Cost  Growth 

Median  Production 

Cost  Growth 

D  Statistic** 

p-value*** 

Full  Set  of  programs 
in  studies 

34 

85% 

66% 

47 

61% 

22% 

28% 

1% 

Full  Set,  outliers 

excluded 

33 

71% 

63% 

45 

56% 

22% 

29% 

1% 

Programs  with  FSD 

start  dates  laterthan 

1974 

25 

55% 

47% 

17 

61% 

20% 

29% 

32% 

Post- 1974  programs, 

outliers  excluded 

24 

49% 

42% 

15 

13% 

10% 

38% 

11% 

*  used  http://www.physics.csbsju.edU/stats/KS-test.n.plot_form.html 
**The  maximum  difference  between  the  data  points  in  the  distributions 

***The  confidence  in  rejecting  the  null  hypothesis  that  the  two  samples  are  drawn  from  the  same  distribution 


For  the  entire  set  of  eighty-one  programs  evaluated  in  the  in  the  two  papers,  the  programs 
using  DTC  had  higher  production  cost  growth  than  did  the  programs  not  utilizing  DTC,  and  the 
differenee  is  statistieally  significant  at  the  7  percent  confidenee  level.  The  exclusion  of  outliers 
resulted  in  little  ehange.  The  programs  starting  after  1975  had  lower,  but  still  signifieant  cost 
growth,  with  or  without  DTC,  and  the  mean  growth  of  the  DTC  programs  was  still  higher  than 
for  non-DTC  programs.  The  p-value  of  32  pereent  indicates,  however,  low  confidence  that  the 
observed  differenee  is  signifieant  (i.e.,  that  a  real  effect  of  DTC  is  being  observed).  The 
exelusion  of  outliers  produeed  an  outcome  even  worse  for  DTC,  with  the  inerease  over  non-DTC 
programs  having  a  higher  level  of  eonfidence  with  a  p-value  of  11  percent.  The  difference 
between  these  results  and  those  presented  in  P-2722  are  probably  explained  by  the  latter’s  use  of 
dollar-weighted  averaging  (the  paper  does  not  provide  the  eost  data  needed  to  confirm  that 
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supposition).  The  bottom  line  is  that  the  available  data  do  not  indicate  that  the  DTC  initiative 
was  sueeessful  in  controlling  costs.  It  may  have  aetually  been  eounter-produetive,  though  that 
eannot  be  said  eonelusively. 

The  periods  of  time  eneompassed  by  the  DTC  and  non-DTC  samples  differed  signifieantly, 
as  would  be  expeeted  since  many  (but  not  all)  of  the  non-DTC  programs  were  initiated  prior  to 
the  implementation  of  the  DTC  policy  in  the  early  1970s.  The  median  start  year  for  the  DTC 
sample  was  1976,  whereas  that  for  the  non-DTC  sample  was  1970.  Nonetheless,  seventeen  of  the 
forty-seven  non-DTC  programs  had  start  dates  of  1975  or  later,  raising  the  question  of  why  those 
programs  did  not  establish  a  DTC  goal. 


Dollar-weighted  average  results  are  less  useful  for  the  current  analysis  because  it  is  focused  on  the  effectiveness 
of  a  program  management  technique,  irrespective  of  program  size. 

The  research  team  has  not  been  able  to  determine  exactly  when  the  formal  requirement  to  establish  a  DTC  goal 
was  put  in  place.  It  was  clearly  in  place  by  1975. 
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Over  the  early  1970s  to  early  1980s  period  many  changes  were  occurring  in  the  DOD 
acquisition  process, and  it  has  been  observed  that  cost  growth  in  defense  acquisition  programs 
decreased  significantly  compared  to  both  earlier  and  later  eras,  as  shown  in  Figure  1 . 
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Figure  1.  Procurement  Cost  Growth  by  Time  Period 


Unfortunately  it  has  not  been  possible  to  establish  definitively  which  changes,  if  any, 
produced  those  encouraging  results.  It  is,  of  course,  difficult  to  isolate  such  effects  without  a 
much  more  in-depth  review  of  the  programs.  Two  other  factors  that  were  at  work  in  that  period 
may  explain  the  better  results — the  advent  of  the  CAIG  and  a  far  more  proactive  office  of  the 
DDR&E  compared  to  either  earlier  or  later  periods.  Two  separate  IDA  publications  have 
addressed  these  topics.  The  first  paper.  Cost  Growth  in  Major  Weapon  Procurement  Programs, 
IDA  Paper  NS  P-3832, shows  improvements  in  cost  growth  in  MDAPs  starting  after  the 
establishment  of  the  CAIG  in  1972,  attributable  at  least  in  part  to  improved  initial  cost  estimates. 
The  second  paper.  What  to  Buy?  The  Role  of  Director  of  Defense  Research  and  Engineering 


See  William  D.  O’Neil,  and  Gene  H.  Porter,  What  to  Buy?  The  Role  of  Director  of  Defense  Research  and 
Engineering  (DDR&E):  Lessons  from  the  1970s,  IDA  Paper  P-4675  (Alexandria,  VA:  IDA,  January  2011). 

This  chart,  from  an  unpublished  IDA  briefing,  was  developed  using  an  updated  version  of  the  database  used  in 
David  L.  McNicoI,  Cost  Growth  in  Major  Weapon  Procurement  Programs,  Second  Edition,  IDA  Paper  NS  P- 
3832  (Alexandria,  VA:  IDA,  October  2004). 

Ibid.,  9,  cites  several  studies  supporting  a  conclusion  that  the  institution  of  independent  cost  estimating  resulted 
in  improved  initial  cost  estimates,  and  thus  to  reducing  cost  growth. 
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(DDR&E):  Lessons  from  the  1970s,  IDA  Paper  P-4675"^^  compares  Office  of  the  Secretary  of 
Defense  (OSD)  acquisition  management  and  oversight  practices  in  of  the  1970s  to  those  of  the 
1990s  and  later  and  notes  that  better  outcomes,  including  lower  cost  growth,  were  achieved  in 
the  mid-late  1970s.  The  point  of  this  discussion  is  that  other  factors  are  more  likely  to  have 
contributed  to  improved  acquisition  program  outcomes  in  the  DTC  era — a  conclusion  supported 
by  the  statistical  analysis  cited  above. 


C.  Cost  as  an  Independent  Variable 

In  the  mid-1990s,  anticipating  new  challenges  and  diminished  budgets  following  the  end  of 
the  cold  war,  DOD  instituted  a  comprehensive  program  of  acquisition  reform. As  that 
movement  developed  over  the  decade  it  came  to  encompass  some  sixty-three  initiatives."^^  CAIV 
is  one  among  the  sixty-three,  but  it  was  not  simply  a  face  in  the  crowd;  statements  by  top 
acquisition  officials  made  it  clear  that  they  regarded  CAIV  as  among  the  most  important  of  the 
five  dozen  initiatives. 

The  first  codified  guidance  for  CAIV  was  in  DODD  5000.1,  Defense  Acquisition,  published 
in  March  1996.  That  publication  was  preceded  by  a  memorandum  from  Under  Secretary  Paul 
Kaminski  implementing  the  recommendations  of  a  working  group  he  established  under  the 
auspices  of  the  Defense  Manufacturing  Council  and  charged  to  “address  approaches  and 
measures  to  reduce  life  cycle  costs.”  Among  several  measures,  Kaminski  identified  the  need  for 

aggressively  managing  programs  to  meet  those  objectives,  thus  making  cost  a 
major  driver  while  meeting  the  true  needs  of  the  warfighter.  This  concept  is  called 
“cost  as  an  independent  variable”  (CAIV).  CAIV  is  the  DOD  equivalent  of  best 
commercial  business  practices. 

The  memorandum  also  established  eight  acquisition  programs  to  serve  as  “flagships”  for 
CAIV  implementation.  Using  readily  available  materials,  primarily  SARs  and  earlier  IDA 
publications,  the  current  research  team  conducted  a  case  study  analysis  of  the  CAIV  flagship 
programs  along  with  four  other  contemporary  programs  that  prominently  attempted  to  employ 
CAIV.  These  case  studies  are  reported  in  Appendix  A  and  summarized  at  the  end  of  this  chapter. 

Although  CAIV  figured  prominently  in  issuances  of  the  DOD  5000  documents  through 
April  2002,  the  May  2003  issuance  of  DODD  5000.1,  The  Defense  Acquisition  System,  contained 
only  a  vestigial  reference  to  CAIV,  and  the  DODI  5000.2,  Operation  of  the  Defense  Acquisition 
System,  issued  at  the  same  time  contained  no  reference  to  CAIV.  The  May  2003  version  of 
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See  O’Neil,  and  Porter,  IDA  P-4675. 

William  J.  Perry,  [Secretary  of  Defense],  “Acquisition  Reform:  A  Mandate  for  Change,”  9  Feb  1994. 

Christopher  H.  Hanks  et  al..  Reexamining  Military  Acquisition  Reform:  Are  We  There  Yet?  MG-291  (Santa 
Monica,  CA:  RAND  Arroyo  Center,  2005),  5-13. 


Paul  Kaminski,  Memorandum  for  Distribution  Subject:  Reducing  Life-Cycle  Costs  for  New  and  Fielded 
Systems,  December  4,  1995.  No  copy  of  this  memo  has  been  found  but  the  text  of  it  is  at 


i://www.navair.navy.mil/nawctsd/Resources/Library/Acqguide/caainvar.htm. 
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DODD  5000.1  was  reissued  in  2007"^’  with  the  CAIV  language  still  in  plaee  and  remains  in 
effeet  as  of  this  writing. 

The  original  CAIV  provision  in  the  1996  DODD  5000.1  was: 

Cost  as  an  Independent  Variable  (CAIV).  Fiseal  eonstraint  is  a  reality  that  all 
partieipants  in  the  defense  aequisition  proeess  must  reeognize.  Cost  must  be 
viewed  as  an  independent  variable.  Aeeordingly,  aequisition  managers  shall 
establish  aggressive  but  realistie  objeetives  for  all  programs  and  follow  through 
by  trading  off  performanee  and  sehedule,  beginning  early  in  the  program  (when 
the  majority  of  eosts  are  determined),  to  aehieve  a  balaneed  set  of  goals,  based  on 

48 

guidanee  from  the  MDA. 

The  aecompanying  DOD  5000. 2-R,  Mandatory  Procedures  for  Major  Defense  Acquisition 
Programs  (MDAPs)  and  Major  Automated  Information  System  (MAIS)  Acquisition  Programs, 
whieh  was  published  at  the  same  time,"^^  ineluded  mueh  more  detailed  guidanee  in  an  entire 
seetion  (3.3.3)  on  CAIV.  At  Milestone  I  (entry  into  Program  Definition  and  Risk  Reduction 
(PDRR) — roughly  equivalent  to  Milestone  A  today)  the  MDA  was  to  approve  “CAIV 
objectives.”  The  basic  philosophy  is  stated  thusly: 

The  acquisition  strategy  shall  address  methodologies  to  acquire  and  operate 
affordable  systems  by  setting  aggressive,  achievable  cost  objectives  and  managing 
achievement  of  these  objectives.  Cost  objectives  shall  be  set  to  balance  mission 
needs  with  projected  out-year  resources,  taking  into  account  anticipated  process 
improvements  in  both  DOD  and  defense  industries,  (emphasis  added). 

The  main  conceptual  difference  between  the  approach  in  DOD  5000. 2-R  and  current 
guidance  is  the  word  “aggressive.”  The  guidance  in  DOD  5000. 2-R  is  further  elaborated  as 
follows: 

[At  Milestone  I  the  program  manager]... shall  have  established  life-cycle  cost 
objectives  for  the  program  through  consideration  of  projected  out-year  resources, 
recent  unit  costs,  parametric  estimates,  mission  effectiveness  analysis  and  trades, 
and  technology  trends.  A  complete  set  of  life-cycle  cost  objectives  shall  include 
RDT&E,  production,  operating  and  support,  and  disposal  costs. 

This  provision  makes  it  clear  that  CAIV  cost  objectives  include  O&S  costs  and  should  be 
developed  based  on  projected  fiscal  resources. 


And  renamed  DODD  5000.01.  http://www.dtic.mil/whs/direetives/eorres/pdf/500001p.pdf. 

DOD,  The  Defense  Acquisition  System,  DODD  5000.1  (Washington,  DC:  DOD,  15  March  1996),  5,  paragraph 
D.l.f 

There  was  no  DODI  5000.2  promulgated  at  that  time.  The  February  1991  version  of  DODI  5000.2  was 
cancelled  by  the  1996  issuance  of  DODD  5000.1. 

DOD,  Mandatory  Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  and  Major  Automated 
Information  System  (MAIS)  Acquisition  Programs,  DODR  5000. 2-R  (Washington,  DC:  DOD,  15  March  1996), 
6,  part  3. 

Ibid.,  7,  part  3. 
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The  DODD  5000.1  published  on  Oetober  23,  2000  expanded  the  CAIV  language  of  the 
1996  version: 

4.5.2.  Cost  and  Affordability.  Fiseal  eonstraint  is  a  reality  that  all  participants  in 
the  acquisition  system  must  recognize.  Cost  must  be  viewed  as  an  independent 
variable,  and  the  DOD  Components  shall  plan  programs  based  on  realistic 
projections  of  funding  likely  to  be  available  in  future  years.  To  the  greatest  extent 
possible,  the  DOD  Components  shall  identify  the  total  costs  of  ownership,  and  at 
a  minimum,  the  major  drivers  of  total  ownership  costs.  Consistent  with  the 
Chairman  of  the  Joint  Chiefs  of  Staff  guidance  on  requirements  generation,  the 
user  shall  treat  cost  as  a  military  requirement  and  state  the  amount  the  Department 
should  be  willing  to  invest  to  obtain,  operate,  and  support  the  needed  capability 
over  its  expected  life  cycle.  Acquisition  managers  shall  establish  aggressive  but 
realistic  objectives  for  all  programs  and  follow  through  by  working  with  the  user 
to  trade  off  performance  and  schedule,  beginning  early  in  the  program  (when  the 
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majority  of  costs  are  determined). 

This  issuance  cancelled  DOD  5000. 2-R;  however,  a  memorandum  was  signed  stating  that  DOD 
5000.2-R  would  remain  in  effect  as  interim  guidance  until  a  revision  was  published.  When  DODl 
5000.2  was  reissued  in  April  2002,  the  only  reference  to  CAIV  was 

4. 7.2. 1.4.  The  user  or  user’s  representative  shall  work  with  the  Program  Manager 
or  other  system  developer  (e.g.,  the  Demonstration  Manager  for  Advanced 
Concept  and  Technology  Development  projects)  to  establish  and  refine  cost  as  an 
independent  variable  (CAlV)-based  cost  and  performance  objectives  and  critical 
schedule  dates.  The  CAlV-based  parameters  and  critical  schedule  dates  shall  also 
be  included  in  the  APB.^^ 

However,  DOD  5000.2-R  was  also  reissued  at  that  time  with  essentially  the  same  extensive 
CAIV  provisions  as  the  previous  issuance.  (DODD  5000.1  was  not  reissued  and  remained  in 
effect.) 

A  dramatic  turn  away  from  CAIV  was  evident  with  the  May  2003  reissuance  of  both 
DODD  5000.1  and  DODI  5000.2.  A  single  reference  to  CAIV  is  found  in  DODD  5000.1  but 
none  whatsoever  in  DODI  5000.2,  and  the  latter  explicitly  cancelled  DOD  5000.2-R  with  its 
extensive  CAIV  discussion.  The  DODD  5000.3  of  May  2003  was  reissued  in  2007  including 


DOD,  The  Defense  Acquisition  System,  DODI  5000.1  (Washington,  DC:  DOD,  23  October  2000),  8. 

DOD,  Operation  of  the  Defense  Acquisition  System,  DODI  5000.2  (Washington,  DC:  DOD,  5  April  2002),  14. 
APB — Acquisition  Program  Baseline.  This  provision  occurs  several  times  in  the  DODI  5000.2  and  DOD 
5000.2-R  documents  of  the  period;  however,  the  research  team’s  review  the  APB  SARs  for  the  CAIV  flagship 
programs  in  the  DAMIR  revealed  no  instances  of  recorded  CAIV  objectives. 

Ibid.,  l.A  January  2002  memorandum  signed  by  USD(AT&L)  E.C.  Aldridge  (in  office  from  May  2001  to  May 
2003)  emphasized  CAIV:  “I  have  approved  a  metric  to  require,  by  the  end  of  FY02,  100%  of  defense  programs 
to  incorporate  a  cost-as-an-independent-variable  (CAIV)  plan...”  Since  this  memo  seems  to  indicate  a  buy-in  of 
the  CAIV,  it  is  all  the  more  curious  that  only  seventeen  months  later  that  commitment  vanished  entirely. 
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its  single  CAIV  reference,  and  remains  in  effect  as  of  this  writing.  The  language  in  the 
2003/2007  DODD  5000.1  regarding  CAIV  is  as  follows: 

El. 4.  Cost  and  Affordability.  All  participants  in  the  acquisition  system  shall 
recognize  the  reality  of  fiscal  constraints.  They  shall  view  cost  as  an  independent 
variable,  and  the  DOD  Components  shall  plan  programs  based  on  realistic 
projections  of  the  dollars  and  manpower  likely  to  be  available  in  future  years.  To 
the  greatest  extent  possible,  the  MDAs  shall  identify  the  total  costs  of  ownership, 
and  at  a  minimum,  the  major  drivers  of  total  ownership  costs.  The  user  shall 
address  affordability  in  establishing  capability  needs. (emphasis  added) 

This  provision  can  only  be  viewed  as  opaque  by  DOD  acquisition  practitioners  unfamiliar  with 
its  history. 

1.  CAIV  versus  DTC 

Inevitably,  some  have  wondered  what  was  really  new  about  CAIV,  and  several  authors 
attempted  to  draw  a  distinction  between  it  and  DTC,  in  some  cases  displaying  ignorance  of  the 
actual  policies  under  DTC.^^  Within  the  context  of  the  several  acquisition  reform  initiatives 
undertaken  in  same  time  period  as  CAIV,  one  early  review  concluded  that 

the  basic  concept  of  CAIV  is  not  dramatically  different  from  the  “Design  to  Cost” 
concept  applied  with  mixed  results  in  the  early  1970s.  The  difference,  according 
to  advocates,  is  that  CAIV  is  being  implemented  in  an  environment  of  much  more 
profound  change  to  the  traditional  acquisition  system  and  culture.  They  believe 
that  this  gives  it  a  much  greater  chance  of  success. 

However,  one  could  legitimately  question  whether  the  1990s  was  a  period  of  “much  more 
profound  change  to  the  traditional  acquisition  system  and  culture”  than  the  early  1970s,  which 
was  regarded  by  contemporaries  as  a  period  of  radical  transformation  and  the  focus  of  top  DOD 

CO 

management  on  acquisition  issues  arguably  reached  its  zenith. 

The  reliance  on  integrated  product  teams  (IPT),  and  in  particular  on  a  Cost/Performance 
IPT  (CPIPT),  is  sometimes  cited  as  a  distinctive  feature  of  CAIV,  as  contrasted  to  DTC. 
However,  IPTs  had  been  widely  advocated  and  employed  since  the  1980s,  including  in  programs 
subject  to  DTC,  in  the  experience  of  one  of  the  authors  of  this  paper.  Moreover,  developer-user 


DOD,  The  Defense  Acquisition  System,  DODD  5000.1  (Washington,  DC:  DOD,  12  May  2003),  5,  Enclosure  1. 

Stacy  S.  Azama,  “Cost  as  an  Independent  Variable  (CAIV):  Teaching  Note,”  Defense  Systems  Management 
College,  Funds  Management  Department,  June  2000.  Higgins,  Guy,  “CAIV — An  Important  Principle  of 
Acquisition  Reform,”  Program  Manager  26,  No.  1  (Jan-Feb  1997):  44-47.  Kausal,  B.  A.  “Tony”,  IV, 
“Controlling  Cost — A  Historical  Perspective,”  Program  Manager  25,  No.  5  (Nov-Dec  1996):  22-28. 

Mark  A.  Lorell,  etal..  Cheaper,  Faster,  Better?  Commercial  Approaches  to  Weapons  Acquisition,  MR-1147-AF 
(Santa  Monica,  CA:  RAND  Corp.,  2000),  22,  fn.  18. 

O’Neil,  and  Porter,  IDA  P-4675,  49-54. 
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teams  that  were  funetionally  equivalent  to  CPIPTs  were  employed  during  World  War  II,  with 
exeellent  results. 

2,  CAIV  and  Should-Cost 

Perhaps  even  more  than  for  DTC,  CAIV  bore  similarities  to  the  eurrent  should-eost 
initiative.  The  Defense  Manufaeturing  Council  working  group  report  that  resulted  in 
implementation  of  CAIV  states: 

Aggressive  cost  objectives  means  costs  objectives  that  are  the  DOD-equivalent  of 
sound  commercial  business  practices.  These  objectives  will  be  set  as  early  as 
possible  (e.g.,  Milestone  I  or  before  for  most  systems).  It  is  expected  that  these 
objectives  will  be  much  lower  than  would  be  projected  for  a  system  using  past 
ways  of  doing  business. 

This  statement  encapsulates  the  idea  that  initial  cost  estimates  derived  from  cost  estimating 
relationships  in  prior  defense  acquisition  programs  (and  virtually  all  of  them  are  so  derived)  are 
based  intrinsically  on  the  “old  ways  of  doing  business”  and  that  those  costs  can  be  reduced  by 
sound,  aggressive  management  actions.  Even  though  the  report  does  state  that  PMs  should  not  be 
penalized  for  “failures  that  might  occur  despite  best  management  efforts,”  it  also  states  that 
“promotion  policies,  awards  and  other  formal  recognition  are  important  in  providing  feedback 
that  jobs  have  indeed  been  done  well.”^^  The  clear  implication  is  that  success  in  implementing 
CAIV  should  lead  to  such  rewards.  The  danger  of  this  approach  is  that  managers  will  be 
motivated  to  attempt  “heroic”  measures  to  dramatically  reduce  costs.  There  is  some  evidence  that 
such  outcomes  occurred  in  some  of  the  CAIV  flagship  programs,  such  as  JASSM  and  SBIRS, 
and  possibly  in  the  non-flagship  CAIV  programs  such  as  EFV  and  ECS.  See  Section  2.D  for  a 
detailed  discussion  of  this  point. 

3,  CAIV  and  Commercial  Practices 

One  of  the  principal  arguments  for  CAIV  was  that  it  mirrored  the  practice  in  commercial 
industry,  which  was  presumed  to  be  especially  efficient  and  economical  in  its  operations.  This  is 
an  oversimplification  and  analyzing  it  reveals  the  limitations  of  CAIV. 

In  industries  such  as  consumer  electronics,  consumer  durables,  and  industrial  catalog  goods, 
many  firms  sell  in  markets  whose  structure  approximates  price-based  competition.  In  such 
markets  development  and  manufacturing  costs  generally  are  a  substantial  fraction  of  selling 
price,  and  firms  need  to  be  able  to  develop  new  models  to  meet  a  price  point,  using  CAIV-like 


Ibid.,  13-14. 

IDA  has  not  been  able  to  locate  an  image  copy  of  either  the  Kaminski  memo  or  the  attached  Defense 
Manufacturing  Council  Working  Group  Report.  However,  an  HTML  version  is  available  at 
http://www.navair.navv.mil/nawctsd/Resources/Librarv/Acqguide/caivwgp.htm. 

Ibid.,  page  number  unknown  since  original  document  has  not  been  located. 
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techniques.  Developing  a  new  model  for  an  established  line  of  automobiles,  home  entertainment 
systems,  or  refrigerators,  however,  is  a  task  that  is  very  little  like  that  of  the  typical  MDAP. 

Occasionally  firms  in  these  lines  of  business  seek  to  develop  an  entirely  new  class  of  good 
to  meet  an  emergent  need  or  newly  available  technology,  a  task  more  like  that  involved  in  an 
MDAP,  although  typically  on  a  smaller  scale.  Although  these  cases  vary,  frequently  the 
development  objectives  place  substantially  greater  emphasis  on  new  kinds  of  performance,  with 
cost  less  rigidly  constrained. 

Moreover,  industrial  firms  often  find  CAIV-like  strategies  much  more  challenging  than 
CAIV  advocates  recognized,  particularly  in  industries  producing  goods  most  comparable  in  scale 
and  complexity  to  MDAPs,  such  as  commercial  aircraft.  The  Boeing  787  “Dreamliner”  is  a  well- 
known  example  of  a  project  whose  costs  outstripped  its  maker’s  intentions.  Sources  in  the 
commercial  transport  industry  confirm  that  throughout  the  jet  age  few  airliner  projects  have  met 
cost  goals.  The  industry  as  a  whole  has  survived  because  demand  growth  has  exceeded 
expectations,  a  critical  factor  in  an  industry  where  learning  curves  remain  relatively  steep.  But 
where  there  were  once  more  than  a  dozen  manufacturers  of  large  airliners  in  the  West,  today 
there  are  only  two.  In  almost  every  case,  exits  from  the  market  or  mergers  were  forced  by 
unexpected  cost  growth. 

In  many  cases  firms  are  able  to  suppress  news  of  serious  cost  growth,  out  of  fear  for  the 
effect  on  stock  prices  or  on  customer  response.  In  one  industry,  however,  special  circumstances 
have  permitted  a  knowledgeable  analysis  of  project  cost  growth,  with  very  interesting  results. 
Edward  R.  Merrow  heads  a  consultancy  that  serves  clients  in  the  petroleum  and  petrochemical 
industry.  His  firm  has  been  able  to  amass  a  very  detailed  and  consistent  data  set  covering  a  great 
many  projects,  and  he  has  published  an  analysis  of  “megaprojects,”  meaning  projects  costing  in 
excess  of  one  billion  dollars.  Many  of  these  have  a  larger  share  of  what  in  DOD  would  be 
classified  as  MILCON  expense  than  the  typical  MDAP,  and  on  the  whole  they  involve  less  of 
technology  advancement  than  the  average  MDAP.  In  terms  of  size  and  complexity,  however, 
they  are  broadly  comparable  to  MDAPs. 

Merrow  doesn’t  share  the  details  of  his  data  set,  but  he’s  a  competent  and  experienced 
econometrician  with  an  academic  background  whose  analysis  appears  entirely  consistent 
internally.  The  results  reported  in  the  book  build  on  and  are  consistent  with  a  substantial  body  of 
earlier  work  which  was  reported  in  greater  detail.  He  provides  a  clearer,  fine-grain  picture  of 
the  roots  of  cost  growth  than  has  ever  been  reported  for  any  MDAP. 


Edward  W.  Merrow,  Industrial  Megaprojects:  Concepts,  Strategies,  and  Practices  for  Success  (Hoboken,  NJ: 
John  Wiley  &  Sons,  2011).  Also  relevant  is  Edward  W.  Merrow,  “Why  Large  Projects  Fail  More  Often: 
Megaproject  Failures:  Understanding  the  Effects  of  Size,”  annotated  briefing  (Ashburn,  VA:  Independent 
Project  Analysts,  Inc.,  2011). 

Edward  W.  Merrow,  Understanding  the  Outcomes  of  Megaprojects:  A  Quantitative  Analysis  of  Very  Large 
Civilian  Projects,  R-3560-PSSP  (Santa  Monica,  CA:  RAND  Corp.,  1988);  Edward  W.  Merrow,  Kenneth  E. 
Phillips,  and  Christopher  W.  Myers,  Understanding  Cost  Growth  and  Performance  Shortfalls  in  Pioneer 
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All  of  the  projects  Merrow  reports  on  had  cost  objectives,  and  cost  was  of  crucial 
importance  in  every  case,  as  it  was  the  denominator  in  return  on  investment  (ROI).  Since  each 
project  was  so  large,  a  poor  ROI  could  threaten  the  profitability  of  the  firm  undertaking  it,  or 
even  the  firm’s  viability.  He  says  that,  in  general,  cost  growth  of  25  percent  will  wipe  out  a 
project’s  ROI  and  that  greater  growth  will  result  in  negative  ROI;  for  this  reason  he  takes  25 
percent  growth  as  a  critical  threshold. 

Figure  2  compares  the  proportion  of  projects  with  acquisition  costs  exceeding  25  percent 
cost  growth  as  reported  by  Merrow  for  industry  megaprojects  with  that  for  105  MDAPs  initiated 
between  1971  and  2000,  calculated  from  the  Cost  Assessment  and  Program  Evaluation  (CAPE) 
refined  SAR  data  set.  Questions  regarding  data  comparability  counsel  some  caution  in  evaluating 
this  comparison,  but  clearly  there’s  nothing  here  to  support  a  hypothesis  of  superior  cost  control 
in  commercial  industry,  given  comparable  challenges. 
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Figure  2.  Comparative  Large-project  Acquisition  Cost  Growth 


Process  Plants,  R-2569-DOE  R-2569-DOE  (Santa  Monica,  CA:  RAND  Corp.,  1981);  Edward  W.  Merrow, 
Stephen  W.  Chapel,  and  Christopher  Worthing,  Review  of  Cost  Estimation  in  New  Technologies:  Implications 
for  Energy  Process  Plants,  R-2481-DOE  R-2481-DOE  (Santa  Monica,  CA:  RAND  Corp.,  July  1979). 
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4.  Obstacles  to  Evaluating  the  Efficacy  of  CAIV 

In  order  to  draw  lessons  for  the  future  from  the  CAIV  experience,  it  would  be  helpful  to 
understand  CAIV’s  efficacy  in  constraining  costs.  The  resources  of  this  research  restrict  the 
ability  to  do  so,  but  beyond  that  there  are  inherent  uncertainties  and  situational  limits  that  must 
be  recognized.  These  obstacles  include 

•  Overdetermination.  To  evaluate  the  effectiveness  of  CAIV,  one  must  separate  the 
effects  of  CAIV  from  those  of  other  essentially  coterminous  acquisition  policy 
initiatives,  many  of  which  also  have  controlling  costs  as  at  least  one  objective. 

•  Persistent  practices.  Essentially,  CAIV  codified  existing  “best  practice”  rather  than 
instituting  wholly  new  and  untried  doctrines.  This  substantially  reduced  the  risks 
associated  with  its  introduction  but  at  the  same  time  meant  that  its  introduction  did  not 
represent  a  sharp  change  in  practices  at  a  defined  moment. 

•  Long  lags.  It  can  well  take  twenty  years,  or  even  longer,  for  the  actual  acquisition  costs 
associated  with  an  MDAP  to  become  accurately  known. 

•  Small  sample.  The  actual  number  of  MDAPs  that  are  known  to  have  implemented 
CAIV  remains  comparatively  small. 

Econometrics  offers  statistical  techniques  aimed  at  overcoming  such  limitations  in  order  to 
estimate  the  effect  of  a  single  factor  such  as  CAIV;  however,  the  sample  size  available  for 
programs  using  CAIV  is  too  small  to  support  such  an  analysis.  Thus,  a  case  study  approach 
was  used  to  produce  useful  information  within  the  resources  available  for  this  project.  If 
sufficient  resources  do  become  available,  it  may  be  possible  to  build  a  database  to  support  an 
econometric  analysis  to  test  and  validate  the  conclusions  drawn  from  the  case  studies. 

D.  Analysis  of  Selected  CAIV  Programs 

CAIV  was  in  use  for  a  much  shorter  time  than  DTC;  nonetheless,  it  was  deeply  entrenched 
in  DOD  regulations  from  1996  to  2002.^^  Eight  CAIV  “flagship”  programs  were  named  and  a 
number  of  other  acquisition  programs  purported  to  use  the  methodology.  Unlike,  DTC,  however, 
there  was  never  a  systematic  recording  of  CAIV  cost  objectives  in  SARs,  and  for  that  reason 
among  others,  no  database  exists  like  the  one  compiled  by  IDA  in  the  1989-92  period  for  DTC. 
Thus  this  research,  lacking  the  resources  to  construct  such  a  database,  depended  instead  on  case 
study  analyses  of  the  eight  flagship  programs  plus  four  other  programs  known  to  have  attempted 


Unlike  with  DTC,  the  research  team  did  not  have  a  way  of  determining  the  extent  to  which  CAIV  principles 
were  actually  applied  for  programs  under  development  during  the  CAIV  era.  In  any  case,  the  sample  size  would 
obviously  be  much  smaller  than  for  DTC  because  the  time  period  over  which  the  policy  was  active  was 
considerably  shorter. 

And  as  noted  earlier,  CAIV  exists  in  a  vestigial  form  in  DODD  5000.1  published  in  2007,  still  current. 
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to  employ  the  CAIV  approaeh.^^  The  CAIV  case  studies  (Appendix  A)  include  reviews  of  the 
systems  identified  in  Table  2. 


These  programs  include  the  San  Antonio  class  LPD-17,  Expeditionary  Fighting  Vehicle,  and  Joint  Direct  Attack 
Munition. 
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Table  2.  CAIV  Acquisition  Programs  Analyzed 


Program  Short 

CAIV 

Name 

Flagship? 

Program  Long  Name 

Crusader 

Yes 

Crusader  Advanced  Fieid  Artiiiery  System 

Army 

ATACMS/BAT 

Yes 

Army  Tacticai  Missiie  System/  Briiiiant  Anti- 
Tank 

MIDS 

Yes 

Muitifunctionai  Information  Distribution 

System 

Navy 

AIM-9X 

Yes 

Sidewinder  Air  Intercept  Missile 

LCS 

No 

Littoral  Combat  Ship 

LPD-17 

No 

San  Antonio  Class  Amphibious  Transport 

Dock  Ship 

Marine  Corps 

EFV 

No 

Expeditionary  Fighting  Vehicle 

JDAM 

No 

Joint  Direct  Attack  Munition 

SBIRS 

Yes 

Space-based  Infrared  System 

Air  Force 

JASSM 

Yes 

Joint  Air-to-Surface  Standoff  Missile 

JSF/F-35 

Yes 

Joint  Strike  Fight  (Lightening  II) 

EELV 

Yes 

Evolved  Expendable  Launch  Vehicle 

Table  3  displays  cost  growth  in  the  programs  analyzed,  derived  using  Average  Unit 
Procurement  Costs  (APUC)  from  SARs.  It  can  be  seen  that  that  only  one  (MIDS)  of  the  eight 
CAIV  flagship  programs  was  fully  successful  in  controlling  procurement  cost  growth. 
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Table  3.  Average  Unit  Procurement  Cost  Growth  of  Selected  CAIV  Programs 

Original  Estimate  Current  Estimate  Percent  Change 


JDAM  ($K) 

31 

19.8 

-36% 

JASSM*  ($K) 

399 

703 

76% 

MIDS*  ($K) 

309 

243 

-21% 

AIM-9X*  ($K) 

193 

221 

15% 

SBIRS*  ($M) 

694 

2,775 

300% 

JSF*  ($M) 

50.9 

85.8 

69% 

EELV*  ($M) 

65 

298 

358% 

Crusader*  ($M) 

10.8 

11.5 

6% 

ATACMS/BAT*  ($K) 

72 

118 

64% 

LCS  ($M) 

225 

485 

116% 

LPD-17  ($M) 

743 

1,304 

76% 

EFV  ($M) 

5.3 

13.1 

147% 

Note:  *CAIV  Flagships;  $K  =  thousands,  $M  =  millions 


Four  of  the  case  studies  were  facilitated  by  previous  IDA  research  that  conducted  detailed 

fn 

analyses  of  the  causes  of  cost  growth  for  nine  important  acquisition  programs.  The  following 
paragraphs  summarize  the  findings  of  the  case  studies  presented  in  Appendix  A. 

JDAM.  JDAM  is  an  all-weather  glide-bomb  kit  that  can  be  fitted  to  commodity  bomb 
bodies  to  provide  a  moderate-precision,  relatively  inexpensive,  guidance  system.  Although  not  a 
CAIV  flagship  program,  JDAM  began  as  a  “Defense  Acquisition  Pilot  Project”  with  a  strong 
cost  control  focus  and  was  influential  in  the  emergence  of  the  CAIV  concept.  The  principal 
Military  Service  (hereafter  Service)  using  JDAM  saw  it  as  a  mass-use  weapon  and  was  willing  to 
be  flexible  regarding  performance  to  achieve  low  cost.  Limited  performance  requirements  made 
extensive  use  of  high-volume,  commercial-grade  parts  possible,  and  acquisition  rules  were 
tailored  to  ease  their  use.  The  contract  structure  provided  strong  incentives  for  cost  control,  with 
a  realistic  risk/reward  calculus,  aided  by  large  production  volumes  and  steep  learning  curves. 
The  program  has  been  highly  successful  in  both  cost  and  performance.  By  any  measure,  JDAM 
was  a  signal  success  for  the  CAIV  concept. 

Crusader  (Flagship).  This  system  to  replace  the  M-109  self-propelled  howitzer  suffered  to 
an  unusual  degree  from  requirements  changes,  which  greatly  increased  development  costs.  It  was 
ultimately  cancelled  by  Secretary  Donald  H.  Rumsfeld  in  May  2002,  prior  to  Milestone  II, 
because  of  its  lower  priority  in  the  post-9/11  environment.  While  the  1998  SAR  claimed  a  35 
percent  production  cost  reduction,  in  fact  that  reduction  took  place  before  Milestone  I  in  1994 


Gene  H.  Porter  et  al.,  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  IDA  Paper  P-453 1 
(Alexandria,  VA:  IDA,  December  2009). 
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and  before  its  designation  as  a  CAIV  flagship.  Sinee  no  independent  eost  estimate  based  on  a 
produetion  design  was  ever  eompleted,  the  program  should  be  eonsidered  a  “no  test”  from  CAIV 
sueeess  standpoint. 

ATACMS/BAT  (Flagship).  This  “brilliant”  munition  program  faeed  unpreeedented  and 
highly  ehallenging  teehnieal  performanee  goals,  and  the  struggle  to  meet  them  appears  to  have 
erowded  out  any  serious  eost-eontrol  efforts.  The  program  was  eaneelled  (by  the  Army)  in  late 
2003  beeause  of  teehnieal  diffieulties  evideneed  by  repeated  test  failures.  No  reeognition  of  the 
program  being  a  CAIV  flagship  was  found  in  its  SARs.  APUC  grew  by  20  pereent  or  more  every 
year  exeept  one.  While  quantities  were  greatly  redueed  in  the  last  SAR  before  program 
eaneellation,  there  is  no  evidenee  that  any  others  adjustments  were  made  to  eontrol  produetion 
eosts  (or  indeed  that  any  sueh  adjustments  were  even  feasible). 

MIDS  (Flagship).  MIDS  is  a  multinational  eooperative  development  program  to  design, 
develop  and  deliver  lightweight  taetieal  information  system  terminals  for  aireraft,  ships,  and 
ground  sites.  Milestone  II  was  in  1993  and  Milestone  III  was  July  2003,  so  the  program  was  in 
the  middle  of  full-seale  development  when  designated  as  a  CAIV  flagship.  The  SARs  make  no 
mention  of  CAIV,  and  neither  the  SARs  nor  other  reports  on  the  program  give  any  hint  of  CAIV- 
type  eost-performanee  tradeoffs.  While  there  were  some  signifieant  sehedule  slippages,  eosts 
remained  under  eontrol.  The  terminals  have  proven  attraetive  to  many  users,  resulting  in  high 
produetion  volumes  and  attendant  unit  eost  deereases.  However,  there  is  no  evidenee  that  the 
outeome  was  a  result  of  the  applieation  of  CAIV. 

AIM-9x  (Flagship).  The  AIM-9X  is  the  latest  variant  in  a  long  line  of  Sidewinder  short- 
range  infrared-guided  air-to-air  missiles.  AIM-9X  aehieved  ineremental  improvements  over 
previous  AIM-9  generations  by  ineorporating  advaneed  subsystems.  Most  of  the  new  subsystems 
had  already  been  demonstrated  in  prototype  form,  so  major  risk-reduetion  had  already  taken 
plaee.  Thus,  it  offered  good  prospeets  for  a  sueeessful,  if  not  partieularly  ehallenging,  CAIV 
program.  In  faet,  long  before  the  advent  of  CAIV,  eost  had  been  a  foeus  of  the  program  sinee  the 
initial  requirement  was  approved  in  1990.  CAIV  was  mentioned  in  the  Deeember  1997  SAR  but 
not  thereafter.  APUC  remained  under  eontrol,  and  in  faet  deereased  somewhat,  through  2004. 
However  in  2005  and  again  in  2009  the  APUC  inereased  about  9  pereent  eaeh  time.  Overall  the 
program  was  a  sueeess,  and  its  eost  growth  was  lower  than  average.  However,  the  researeh  team 
found  no  evidenee  that  CAIV  tradeoffs  of  eost  and  performanee  had  oeeurred. 

SBIRS  (Flagship).  SBIRS  replaees  the  Defense  Support  Program  (DSP)  satellites  to 
provide  early  warning  of  ballistie  missile  launehes.  The  program  sought  to  eombine  major 
performanee  advanees  with  substantial  eost  savings.  Early-phase  systems  engineering  was 
avoided  to  save  eosts,  eontributing  to  initial  eost  estimates  that  were  extremely  unrealistie.  While 


The  figures  in  Table  3  are  not  adjusted  for  quantities.  The  Dec- 12  SAR  quantities  associated  with  the  APUC  is 
twice  that  of  the  Dec-97  SAR.  Adjusted  for  the  quantity  difference,  the  unit  cost  decrease  was  about  10  percent 
(assuming  an  85  percent  production  learning  curve  slope). 
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there  appears  to  have  been  a  genuine  effort  to  utilize  the  CAfV  approaeh,  the  emergenee  of 
serious  teehnieal  issues  soon  after  Milestone  II  quiekly  rendered  the  CAIV  eost  objeetives 
irrelevant.  Government  oversight  was  slashed  to  save  its  eosts,  while  performanee  requirements 
were  informally  inereased.  Contractor  incentives  had  little  effect  because  they  were  far  out  of 
reach.  Requirements  were  eventually  trimmed,  partly  to  save  costs  and  partly  as  a  reflection  of 
technological  reality,  but  not  until  massive  cost  growth  had  already  taken  place.  A  great  deal  of 
growth  came  from  bad  engineering.  SBIRS  represents  a  dramatic  failure  of  CAIV,  teaching  a 
strong  lesson  that  CAIV  cannot  work  for  complex  systems  without  sound  early  systems 
engineering  as  a  basis  for  credible  initial  cost  estimates. 

JASSM  (Flagship).  JASSM  was  initiated  after  the  cancellation  of  the  predecessor  Tri- 
Service  Standoff  Attack  Missile  (TSSAM)  program,  largely  because  of  a  substantial  cost 
overrun.  With  the  same  PM  initially  as  JDAM  and  given  its  success,  there  was  great  optimism 
that  taking  the  same  approach  on  JASSM  would  lead  to  similar  results.  However,  soon  after 
Milestone  II,  costs  began  to  grow.  Inadequate  systems  engineering  contributed  to  late  recognition 
of  reliability  issues,  resulting  in  costly  reworks.  Thus  the  CAIV  strategy  for  JASSM  was 
remarkably  unsuccessful,  not  only  in  failing  to  control  costs  but  also  in  producing  a  missile  with 
significant  effectiveness  issues  as  well.  In  fact,  some  of  the  steps  taken  to  reduce  production 
costs  contributed  to  reliability  failures.  JASSM  demonstrates  yet  again  the  importance  of  an 
initial  cost  estimate  backed  by  sound  systems  engineering  as  a  valid  basis  for  a  CAIV  cost  goal. 
Ironically,  the  current  APUC  for  the  baseline  JASSM  is  remarkably  close  to  early  cost  estimates 
of  about  $700,000  for  a  missile  with  JASSM’s  characteristics,  before  the  ambitious  claims  that 
acquisition  reform  initiatives  (CIAV,  among  others)  could  achieve  a  40  percent  lower  cost 
objective.  If  both  DOD  PMs  and  contractors  are  strongly  incentivized  to  undertake  “heroic” 
efforts  to  reduce  costs,  their  risk-reward  calculus  may  translate  into  excessive  risks.  There  is 
evidence  that  JASSM  is  an  example. 

JSF/F-35  (Flagship).  F  -35  is  a  strike  fighter  designed  for  the  Air  Force,  Navy,  and  Marine 
Corps,  as  well  as  the  air  forces  in  at  least  eight  allied  nations.  Meeting  the  needs  of  various  users 
resulted  in  three  distinct  versions.  A  variety  of  techniques  were  employed  for  cost  reduction, 
ranging  from  early  competitive  prototyping  to  a  compressed  schedule  with  steep  production 
ramp-up,  but  most  failed  and  some  transformed  into  major  liabilities.  Deficiencies  in  systems 
engineering  and  inadequate  government  oversight  led  to  late  recognition  of  serious  problems.  By 
2003  there  was  a  realization  that  the  designs  were  substantially  overweight  and  that  conventional 
methods  of  weight  control  would  not  alleviate  the  problem.  The  only  solution  was  to  redesign  the 
aircraft  almost  from  scratch,  even  though  assembly  of  production  prototypes  had  already  begun. 
In  the  process  it  was  necessary  to  give  up  many  of  the  features  included  to  save  money. 
Moreover,  it  was  necessary  to  extract  more  power  from  the  engine,  which  further  increased  its 
cost.  The  redesigns  seriously  disrupted  the  tight  schedules  designed  to  reduce  costs.  These 
difficulties  probably  prevented  the  CAIV  approach  from  having  any  real  impact  on  the  JSF/F-35 
program.  Because  of  the  large  numbers  of  aircraft  that  the  F-35  will  replace,  affordability  has 
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always  been  a  major  foeus  of  the  program.  The  big  tradeoff  that  the  Department  was,  obviously, 
not  willing  to  make  was  on  the  short  takeoff/vertieal  landing  (STOVL)  eapability  needed  to 
replaee  the  AV-8B  Harriers  of  both  the  Marine  Corps  and  the  United  Kingdom. 

EELV  (Flagship).  The  EELV  program  began  in  the  early  1990s  with  the  objeetive  of 
assuring  adequate  launeh  serviees  for  both  military  and  intelligenee  payloads.  To  reduee  eosts, 
the  original  plan  was  for  a  hybrid  system  of  launehers  that  eould  meet  both  Government  and 
eommereial  needs.  Savings  projeetions  depended  heavily  on  extremely  unrealistic  market 
projections  that  were  only  slowly  acknowledged,  resulting  in  the  continued  pursuit  of  policies 
that  were  counterproductive  in  the  actual  environment.  The  emergence  of  low-cost  foreign 
competitors  essentially  eliminated  hopes  for  commercial  sales.  Thus,  costs  could  not  be 
controlled  through  competition  and  cost-sharing  as  planned.  Significant  cost-performance 
tradeoffs  were  never  really  feasible.  The  size  mix  of  boosters  needed  was  determined  by  the 
weight  of  the  payloads  to  be  placed  in  orbit,  which  was  completely  outside  of  the  control  of  the 
program.  The  KPP  is  launch  reliability,  and  it  makes  no  sense  to  compromise  on  reliability  and 
risk  destruction  of  expensive  payloads  that  might  be  critical  to  national  security.  It  seems  evident 
that  a  true  CAIV  strategy  was  not  realistic  for  EELV. 

LCS.  The  ECS  was  not  a  CAIV  flagship  program  but  CAIV  was  touted  as  central  to  its 
acquisition  strategy.  The  initial  cost  target  was  directed  by  the  Chief  of  Naval  Operations  (CNO) 
and  was  not  based  on  analysis.  Lack  of  early  program  definition  and  highly  concurrent 
scheduling  led  to  chaotic  development.  Late  imposition  of  a  requirement  for  construction  to 
comply  with  newly-issued  Naval  Vessel  Rules  forced  extensive  redesign  while  construction  of 
the  first  ship  was  in  progress.  The  Navy  relied  heavily  on  acquisition  reforms,  cutting  back 
oversight  and  support.  The  requirements  tradeoffs  that  were  made  reflected  a  significant 
misunderstanding  of  actual  cost  impacts  of  failing  to  recognize  the  full  implications  of  ship 
speed.  By  insisting  on  a  very  high  ship  speed  the  Navy  ensured  high  costs  despite  serious 
sacrifices  in  areas  including  seakeeping,  endurance,  survivability,  supportability,  maintainability, 
and  durability.  The  lesson  is  that  effective  CAIV  application  demands  a  thorough  understanding 
of  both  engineering  as  well  as  operational  factors.  When  it  was  gradually  recognized  that  a  light 
commercial  vessel  could  not  be  expected  to  serve  satisfactorily  as  a  warship,  the  ship  designs 
grew  in  size  and  weight  and  so  did  the  cost  per  ton.  In  summary,  the  Navy’s  early  efforts  to  trade 
performance  and  cost  proved  to  be  highly  unrealistic  and  subsequently  required  major  reversals. 

LPD  17.  This  ship  class  was  designed  to  replace  several  amphibious  ship  types.  The  design 
incorporated  a  sophisticated  command  and  control  suite,  as  well  as  a  variety  of  features  intended 
to  increase  efficiency  and  throughput,  putting  it  on  a  plane  of  complexity  close  to  that  of  surface 
combatants.  The  Navy  made  optimistic  assumptions  about  factors  driving  costs,  due  apparently 
to  budget-constrained  costing.  Then,  both  Navy  and  CAIG  underestimated  the  complexity  of  the 
ship  in  initial  cost  estimates.  Though  not  a  CAIV  program,  there  was  a  heavy  emphasis  on 
limiting  cost  growth.  The  ship  incorporated  novel  systems  and  processes  to  reduce  costs. 
However,  the  contractor  selected  had  no  experience  building  ships  of  this  complexity,  so  these 
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efforts  actually  resulted  in  added  costs  even  though  there  was  no  significant  requirements  growth 
in  the  program.  In  fact,  while  early  designs  traded  away  defensive  features  to  reduce  costs, 
constraints  regarding  the  fundamental  mission  and  interfaces  with  other  systems  left  no  room  for 
further  major  trades.  It  appears  that  a  CAIV  approach  was  seriously  taken,  at  least  early  on.  The 
problems  were  primarily  in  execution,  with  poor  contractor  performance  being  the  biggest  factor. 

EFV.  The  Marine  Corps’  EFV  was  conceived  to  be  an  amphibious  infantry  fighting  vehicle 
capable  of  operating  at  high  speeds,  both  in  the  water  and  ashore.  Performance  requirements 
were  entirely  unprecedented  and  demanded  major  technological  advances.  Because  the 
engineering  issues  were  not  well  understood,  initial  cost  estimates  were  faulty.  In  2009,  late  in 
full-scale  development,  the  vehicle  was  unable  to  complete  OT&E  because  of  severe  reliability 
deficiencies.  EEV  was  deleted  from  the  EY2012  President’s  Budget  request,  with  the 
announcement  of  a  substitute  Amphibious  Combat  Vehicle  (ACV)  to  be  more  affordable  and  far 
less  capable.  While  EFV  was  not  a  CAIV  flagship  program,  in  1998  it  won  the  Packard  Award 
for  acquisition  excellence  in,  among  other  factors,  CAIV  execution.  Early  performance  trades  to 
meet  technical  feasibility  left  little  to  trade  off  later,  once  cost  growth  materialized.  If  the  ACV 
program  does  go  ahead  with  markedly  reduced  performance  goals,  it  will  be,  in  effect,  a  CAIV- 
like  trade  off  of  requirements  to  cut  costs.  Substantial  money  and  time  were  wasted  because 
realistic  cost-constrained  trade  studies  were  not  made  prior  to  full-scale  development.  In  other 
words,  a  CAIV  approach  was  not  really  taken. 

E.  Lessons  from  CAIV  History 

CAIV  was  fundamentally  a  sound  policy.  Its  potential  was  clearly  demonstrated  by  its 
progenitor  program,  JDAM.  Unfortunately,  JDAM  was  the  only  success  that  can  be  reasonably 
attributed  to  CAIV.  It  was  successful  not  only  controlling  but  significantly  reducing  production 
costs  and  was  one  of  the  inspirations  for  the  CAIV  approach.  Compared  to  all  the  other 
programs  examined,  JDAM  was  notable  for  its  simplicity — it  was  the  ideal  CAIV  candidate.  The 
other  three  non-flagship  programs  selected  were  known  in  advance  to  have  had  high  cost 
growth — the  objective  of  the  analysis  was  to  determine  whether  CAIV  was,  in  fact,  used  and  if 
so,  why  it  failed,  if  possible. 

JASSM,  EEEV,  SBIRS,  ATACMS-BAT,  ECS,  EPD  17,  EFV,  and  F-35  (JSF)  have  all 
failed  the  test  of  CAIV  in  varying  ways  and  degrees.  As  with  DTC,  it  could  be  argued  that,  were 
it  not  for  the  use  of  CAIV,  cost  growth  in  these  programs  might  have  been  even  greater.  The  only 
program  for  which  that  case  seems  plausible  is  JASSM.  Unlike  its  predecessor  TSSAM,  JASSM 
at  least  resulted  in  successful  fielding  of  an  important  military  capability  at  an  affordable  cost,  if 
far  above  its  CAIV  target. 


It  was  also  an  inspiration  for  the  belief  that  aggressive  action  on  the  part  of  program  managers  could  achieve 
substantial  cost  reductions,  in  line  with  the  should-cost  approach.  Some  of  the  early  CAIV  literature  spoke  of 
achieving  a  25  percent  reduction  from  “expected”  costs — i.e.,  standard  program  cost  estimates. 


30 


In  one  way  or  another,  the  poor  results  ean  all  be  traeed  in  large  measure  not  to  any  inherent 
defeet  in  the  CAIV  eoneept  but  to  the  limitations  in  how  it  was  applied.  But  to  simply  say  that 
the  deficieneies  were  all  a  matter  of  execution  is  misleading.  While  it  is  easy  to  read  that  as 
implying  that  better  and  more  diligent  execution  by  PMs  would  have  brought  the  desired  results, 
the  above  discussions  largely  belie  that  argument. 

Indeed,  one  of  the  distinctive  features  of  JDAM  was  not  only  that  the  initiating  PM  was 
especially  experienced  and  capable  but  that  he  was  in  a  position  to  exercise  an  unusual  degree  of 
control.  JDAM’s  very  simple  and  limited  interface  with  other  systems,  straightforward 
requirements,  and  the  customer’s  strong  commitment  to  cost  reduction  gave  him  significant 
freedom  for  tradeoffs  between  cost  and  requirements,  which  he  made  full  and  resourceful  use  of. 
Moreover,  the  relatively  uncomplicated  and  small-scale  nature  of  the  product  and  its  production 
processes,  combined  with  the  absence  of  high  urgency  for  delivery,  allowed  the  program  to 
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largely  avoid  concurrency  and  the  “marching  army”  problem  that  accompanies  it. 

The  authorities  given  the  JDAM  program  manager  could  not  to  be  counted  on  in  very  many 
programs.  Indeed,  in  JASSM  the  same  initiating  PM,  with  much  the  same  program  team  and 
applying  many  of  the  same  techniques,  encountered  different  situations  involving  lesser  authority 
at  the  program  level  and,  not  only  had  no  success  in  controlling  costs,  but  produced  a  missile 
with  unacceptable  reliability. 

One  of  the  major  problems  of  CAIV  or  other  policies  in  acquisition  is  that  they  can 
encourage  the  illusion  that  the  processes  they  foster  and  promote — such  as  competition, 
commercial  practices,  prototyping,  IPTs,  financial  incentives,  etc. — have  potency  in  and  of 
themselves.  In  fact,  acquisition  is  like  war  or  entrepreneurial  business;  too  complex,  varied,  and 
mutable  to  yield  to  stereotyped  or  routinized  processes.  The  processes  associated  with  CAIV  are 
tools  that  can  help  achieve  powerful  results  if  used  appropriately  and  imaginatively.  But  they 
have  no  intrinsic  merit  and  their  incorporation  in  the  program  plans  provides  no  inherent 
guarantee  of  any  results,  as  the  case  studies  reviewed  clearly  demonstrate. 

The  case  studies  emphasize  the  wisdom  of  the  CAIV  guidance:  “The  best  time  to  reduce 
life-cycle  costs  is  early  in  the  acquisition  process.  Cost  reductions  shall  be  accomplished  through 
cost/performance  tradeoff  analyses,  which  shall  be  conducted  before  an  acquisition  approach  is 
finalized.”^^ 

The  great  majority  of  the  problems  seen  can  be  traced  largely  or  wholly  to  the  failure  to 
adequately  heed  this  guidance.  The  problem  was  not  simply  unwillingness  to  give  up 

“Once  a  program  enters  full-scale  development,  the  commitment  to  design  for  production  unleashes  a  marching 
army  of  interdependent  engineers  that  needs  to  keep  moving  in  a  tight  formation  through  the  development 
process.  Any  serious  design  problems  that  surface  late  in  the  development  process  can  stop  this  marching  army 
in  its  tracks  at  great  expense  while  the  problem  is  addressed  and  resolved.” — Kendall,  Frank,  “Perspectives  on 
Developmental  Test  and  Evaluation,”  ITEA  Journal  2013;  34:  6-10,  7.  The  term  is  often  used  in  this  sense  in 
referring  to  acquisition. 

DOD,  DOD  5000.2-R  (15  March  1996),  6,  part  3. 
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performance  and  other  requirements  but  lack  of  understanding  and/or  recognition  of  what  the 
cost  and  other  implications  of  the  requirements  truly  were.  It  is  not  realistic  to  characterize  that 
problem  entirely  as  defects  in  cost  estimation.  The  tools  of  cost  estimators  can  work  well  for 
cost-requirements  tradeoffs;  but  if  the  input  to  the  cost  calculation  about  weight,  power,  lines  of 
code,  or  other  crucial  system  sizing  parameters,  is  seriously  flawed  then  the  resulting  cost 
estimate  is  bound  to  be  inaccurate  and  cost  overruns  virtually  built-in. 

The  task  of  determining  the  actual  system  sizing  needed  to  meet  a  given  set  of  proposed 
requirements  is  one  of  early-phase  systems  engineering.  Studies  of  SBIRS,  LCS,  EFV,  and 
JSF/F-35  have  specifically  noted  the  lack  of  early-phase  systems  engineering,  and  no  references 
to  it  have  been  found  with  respect  to  any  of  the  other  systems  researched  here.  Thus,  it  is 
reasonable  to  conclude  that  the  lack  of  early-phase  systems  engineering  was  a  major  factor  in 
inadequate  initial  cost-requirements  tradeoffs  which,  in  turn,  were  the  principal  contributor  to 
cost  “surprises”  in  these  programs.  To  attempt  to  control  costs  without  thorough,  independent, 
and  competent  early-phase  systems  engineering  is  to  make  bricks  without  straw. 

Why  did  these  programs  that  employed  CAIV  fail  to  control  costs?  The  simple  answer  is 
that  in  most  cases  the  programs  were  started  based  on  assumptions  (i.e.,  “framing  assumptions”), 
either  explicit  or  implicit,  that  subsequently  did  not  hold  up.  Except  for  two  programs  cancelled, 
the  programs  reviewed  were  deemed  of  such  importance  that  they  were  restructured — some 
repeatedly — and  continued  under  new  (frequently  implicit)  framing  assumptions.  Clearly 
affordability  goals  and  caps  established  early  in  a  program  are  only  going  to  be  relevant  so  long 
as  the  framing  assumptions  under  which  they  were  established  continue  to  hold.  Furthermore,  it 
is  inevitable  for  the  assumptions  to  change,  sometimes  significantly.  What  is  needed  is  a  process 
that  periodically  re-examines  the  affordability  of  the  entire  investment  program  and  makes  the 
necessary  adjustments  needed  so  that  all  major  acquisition  programs  stay  on  affordable  tracks. 
Such  a  process  was  proposed  in  IDA  Paper  P-4961 . 


According  to  Alexander  Kossiakoff,  a  pioneer  in  the  development  of  systems  engineering  techniques,  “The 
function  of  systems  engineering  is  to  guide  the  engineering  of  complex  systems.”  He  goes  on  to  say  that 
systems  engineering  differs  from  other  disciplines  in  that  it  is  focused  on  the  system  as  a  whole,  is  concerned 
with  customer  needs  and  operational  environment,  leads  system  conceptual  design,  and  bridges  traditional 
engineering  disciplines  and  gaps  between  specialties.  See  Alexander  Kossiakoff  and  William  N.  Sweet,  Systems 
Engineering:  Principles  and  Practice  (Hoboken,  NJ:  Wiley-Interscience,  2003).  The  same  term  is  used  to 
denote  somewhat  different  functions  in  software  engineering,  and  emphasis  in  other  texts  is  often  focused  on 
planning-  and  management-oriented  later-phase  systems  engineering. 

Implementation  of  early  systems  engineering  practices  would  also  help  clarify  initial  framing  assumptions. 
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3.  Affordability  Constraints  for  Operating  and 
Support  (0«&S)  (Sustainment)  Costs 


Effectively  applying  affordability  constraints  for  O&S  costs  to  acquisition  programs  is 
complicated  by  difficulties  in  estimating  and  projecting  O&S  costs  in  a  consistent  and 
meaningful  way.  This  chapter  discusses  some  of  those  difficulties  and  examines  O&S  costs 
reported  in  SARs  to  gain  additional  insights  into  these  issues.  After  surveying  the  background  on 
DOD’s  experience  with  attempting  to  control  O&S  costs  for  acquisition  programs,  the  chapter 
will  give  a  brief  overview  of  problems  in  estimating  O&S  costs.  Then  a  sampling  of  O&S  cost 
estimates  from  SARs  will  be  examined  along  two  dimensions:  comparisons  of  new  systems’ 
O&S  costs  to  antecedent  systems  and  the  trends  over  time  in  reported  O&S  costs.  The  latter  data 
give  some  insights  into  the  range  of  uncertainty  in  early  O&S  cost  estimation. 

As  already  noted,  both  DTC  and  CAIV  included  O&S  costs  within  their  scope  under  the 
rubric  of  life-cycle  costs,  with  CAfV  seeming  to  have  given  them  more  emphasis  than  DTC. 
However,  the  research  team  found  little  evidence  that  either  initiative  made  serious  attempts  to 
actually  control  O&S  costs.  In  fact,  early  DTC  literature  indicates  that  there  was  a  prevailing 
view  that  controlling  O&S  costs  was  “too  hard”  because  valid  O&S  cost  models  did  not  exist. 
The  Defense  Manufacturing  Council  working  group  report  that  launched  CAIV  was  prescient 
regarding  the  difficulty  of  controlling  of  O&S  costs  in  the  development  phase: 

Since  O&S  costs  are  not  easily  measurable  in  the  early  stages  of  the  acquisition 
process,  incentives  to  reduce  O&S  costs  may  require  a  (validated)  model  that 
relates  specific  design  parameters  to  measurable  and  predictable  O&S  costs. 
Reliability  and  maintainability  characteristics,  which  are  more  readily  measured 
and  projected,  might  serve  as  early  indicators  of  progress  towards  meeting  O&S 
cost  objectives.  In  any  event,  DOD  needs  better  cost  models  for  the  O&S  phase  of 
our  programs.  We  face  the  challenge  that  CAIV  may  involve  incentivizing 
savings  and  cost  avoidances  that  will  only  be  realized  in  the  more  distant  future. 

Since  those  historical  initiatives,  the  state  of  the  art  in  O&S  cost  modeling  has  improved, 
though  there  is  wide  agreement  that  the  level  of  fidelity  that  can  be  expected  of  O&S  cost  models 
for  most  defense  systems  is  considerably  below  that  for  modeling  production  costs.  This  is 
important  because  the  use  of  cost  models  is  usually  the  only  plausible  way  to  estimate  the  future 
costs  of  a  new  defense  weapon  system.  They  are  used  to  set  initial  cost  estimates  and  inform  the 


See  Defense  Manufacturing  Council  Working  Group  Report  at 
http://www.navair.navv.niil/nawctsd/Resources/Librarv/Acqguide/caivwgp.htni. 
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establishment  and  traeking  of  affordability  goals  and  eaps.  (See  diseussion  in  Introduction  and  in 
Chapter  2,  Section  2.A.1). 

A.  O&S  Cost  Estimation 

Initial  O&S  cost  estimates  for  an  acquisition  system  are  normally  developed  by  the 
sponsoring  Component’s  cost  analysis  center,  working  closely  with  the  system’s  program  office. 
For  MDAPs,  according  to  law,  independent  cost  estimates  (ICE)  must  be  developed  by  CAPE  at 
Milestone  A  and  Milestone  B.  These  estimates  will  either  corroborate  or  supplant  the  Component 
cost  estimate.  Components  are  required  to  update  their  O&S  cost  estimates  annually  and  report 
the  results  in  SARs. 

Operating  and  support  costs  for  many  already-helded  systems  can  be  obtained  from  the 
DOD  VAMOSC  systems,  which  are  required  by  Congress  to  be  maintained  by  each  Military 
Service.  For  Air  Force  and  Navy  systems,  VAMOSC  is  an  extremely  valuable  source  of  actual 
O&S  costs;  Army  and  Marine  Corps  VAMOSC  systems  are  less  capable. 

The  Operating  and  Support  Cost  Estimating  Guide, published  by  the  Office  of  the 
Director  of  CAPE  (ODCAPE),  is  provided  to  DOD  Components  for  use  in  preparing  O&S  cost 
estimates  developed  for  acquisition  planning.  This  guide  breaks  down  the  O&S  costs  according 
to  a  hierarchical  structure  (see  Figure  3).  The  guide  provides  definitions  of  the  cost  elements,  and 
breaks  them  down  into  even  lower-levels  where  appropriate,  and  discusses  techniques  that  can  be 
used  to  develop  the  estimates.  Making  such  detailed  estimates  for  systems  not  yet  fully 
developed,  much  less,  fielded,  can  be  problematic,  and  the  earlier  the  system  is  in  the  process, 
the  more  problematic.  Analogy  to  existing  systems  is  one  technique  that  might  be  employed  for 
systems  that  exist  only  as  concepts.  VAMOSC  data  for  the  most  similar  existing  system  might  be 
used  as  a  starting  point,  with  adjustments  made  for  known  and  quantifiable  differences  such  as 
crew  size,  or  assumptions  of  reliability  and/or  maintainability  improvements  based  on  experience 
with  other  similar  platforms.  More  subjective  adjustments  for  complexity  might  also  be  made 
based  on  engineering  expertise. 

As  the  design  matures,  equipment  parameters  might  be  well  enough  defined  to  employ  cost 
estimating  relationships  (CER) — in  effect,  a  more  systematic  and  rigorous  application  of  the 
analogy  approach.  CERs  are  developed  based  on  historical  data  that  relate  system  parameters 
such  as  weight,  payload,  range,  etc.  to  operating  cost  parameters,  such  as  maintenance  costs. 
Complexity  factors  such  as  number  of  moving  piece  parts  or  software  lines  of  code  might  be 
related  to  reliability.  Time  could  be  a  parameter  to  reflect  improvements  in  manufacturing 


Background  on  Navy  VAMOSC  is  at 

https://www.  vamose. navv.mil/webpages/general/about.cfm? color=0Q3366. 

Office  of  the  Secretary  of  Defense  (OSD),  Operating  and  Support  Cost-Estimating  Guide  (Washington,  DC: 
OSD,  Cost  Assessment  and  Program  Evaluation,  March  2014).  Note  that  this  guide  is  advisory;  its  use  is  not 
mandated  by  DOD  regulations. 
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tolerances  that  translate  into  reliability  and  maintainability  improvements.  As  the  system 
matures,  more  parameters  become  available  to  use  in  the  CERs,  and  greater  fidelity  can  be 
obtained  by  applying  CERs  at  the  subsystem  level.  Ultimately,  developmental  and  especially 
operational  testing  will  provide  actual  O&S  cost  data  to  improve  or  supplant  model  estimates — a 
process  that  will  continue  through  fielding. 


Second-Level/Thi  rd-Level 

O&S  Cost  Estimating  Structure 

1. 

Unit-Level  Manpower 

1.1  Operations 

1.2  Unit-Level  Maintenance 

4.3  Sustaining/Systems  Engineering 

4.4  Program  Management 

1. 3  Other  Unit-Level 

4.5  Information  Systems 

4.6  Data  and  Technical  Publications 

2. 

Unit  Operations 

2.1  Operating  Material 

4.7  Simulator  Operations  and  Repair 

4.8  Other  Sustaining  Support 

2.2  Support  Services 

2.3  Temporary- DuW 

2.4  Transportation 

5.  Continuing  System  Improvements 

5.1  Hardware  Modifications 

5.2  Software  Maintenance 

3. 

Maintenance 

3.1  Consumable  Materials  and  Repair  Parts 

3.2  Depot  Level  Reparables 

3.3  Intermediate  Maintenance  (External  to 

6.  Indirect  Support 

6.1  Installation  Support 

Unit-Level) 

6.2  Personnel  Support 

3.4  Depot  Maintenance 

6.2.1  Personnel  Administration 

3.5  Other  Maintenance 

6.2.2  Personnel  Benefits 

6.2.3  Medical  Support 

4. 

Sustaining  Support 

6.3  General  Training  and  Education 

4.1  System  Specific  Training 

6.3.1  Recruit  &  Initial  Officer  Training 

4.2  Support  Equipment  Replacement  and 

6.3.2  General  Skill  Training 

Repair 

6.3.3  Professional  Military  Education 

Figure  3.  Cost  Elements  Used  for  O&S  Cost  Estimation 


The  key  data  source  for  the  system  technical  and  programmatic  information  in  an 
independent  cost  estimate  is  the  Cost  Analysis  Requirements  Description  (CARD),^^  which 
provides  a  description  of  the  system  detailed  enough  to  develop  cost  estimates  (acquisition  and 
O&S).  A  CARD  is  required  at  Milestone  A  and  Milestone  and  it  is  the  basis  of  the  ICE.  The 


See  DOD,  DOD  5000. 4-M,  Department  of  Defense  Cost  Analysis  Guidance  and  Procedures  (Washington,  DC: 
DOD.  December  1992). 

DODI  5000.02  specifies:  “Recognizing  that  program  details  are  refined  over  time,  with  fewer  details  available 
for  MDAPs  and  MAIS  programs  approaching  Milestone  A  than  Milestone  B,  DCAPE  will  provide  CARD 
development  guidance  tailored  to  the  specific  review  being  conducted  and  the  type  of  system  being  developed” 
(page  127). 
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CARDS  are  updated  as  the  system  matures.  Prior  to  1992,  when  the  CARD  format  and  reporting 
time  frames  beeame  mandatory,  the  CAPE  reeeived  information  in  a  pieeemeal  fashion.  In 
addition,  the  information  itself  was  subjeet  to  frequent  changes.  This  created  the  potential  for 
giving  less  credence  to  the  O&S  ICE  if  it  was  based  on  data  and  assumptions  that  were  no  longer 
valid.  Without  a  credible  ICE,  it  was  too  difficult  to  manage  O&S  cost  affordability,  so  little 
attention  was  paid  to  it. 

The  MDD  is  the  formal  entry  point  of  an  acquisition  program  into  the  DOD  acquisition 
process.  There  is  no  formal  requirement  for  an  O&S  cost  estimate  for  the  MDD  itself  One  is 
required  immediately  thereafter,  however,  to  support  the  Analysis  of  Alternatives  (AoA).  The 
AoA  is  conducted  during  the  Material  Solution  Analysis  (MSA)  phase  between  the  MDD  and 
Milestone  A  to  compare  the  effectiveness  and  life-cycle  costs  of  the  alternatives  being  evaluated. 
Rough  cost  estimates,  usually  based  on  analogy  with  existing  systems  for  which  O&S  costs  are 
known,  are  developed  for  that  purpose. 

Early  cost  estimates  are  normally  made  by  the  sponsoring  component  cost  analysis 
organization  and  are  typically  based  on  rough  analogies  or,  if  sufficient  data  are  available,  cost 
estimating  relationships.  When  separate  program  offices  are  formed  and  have  preliminary 
designs,  they  may  develop  their  own  cost  estimates  based  on  engineering,  bottom-up, 
approaches.  Those  estimates  will  probably  be  more  detailed  and  more  current.  At  Milestone  B  or 
other  decision  points  the  program  office’s  estimates  should  be  consistent  with  the  updated  ICE. 
However,  over  time,  as  better  data  become  available,  there  may  be  a  divergence.  The  program 
office  estimates,  however,  may  not  reflect  the  official  Service  cost  position. 

B.  O&S  Cost  Affordability 

Uncertainty  in  O&S  cost  estimates  means  that  assessing  affordability  for  O&S  costs  is  more 
challenging  than  for  investment  costs.  In  addition,  there  may  not  be  as  clearly  defined  a  portfolio 
for  O&S  costs  of  a  particular  system  as  for  investment  costs.  (This  is  particularly  true  of  systems 

o  1 

for  ground  forces.  )  O&S  costs  depend  on  numerous  operational  and  other  exogenous 
parameters  that  are  difficult,  even  impossible,  to  estimate  with  much  certainty.  These  include  the 
future  operating  tempo  (OPTEMPO)  for  forces,  training  activities,  manning  levels,  repair  rates, 
overhaul  rates,  maintenance  man-hours  required,  spares  and  fuel  costs,  to  name  a  few.  It  is  also 
difficult  to  apportion  indirect  O&S  costs  to  specific  systems  when  they  reside  in  units  with  many 
other  types  of  systems. 


Formerly  the  Cost  Analysis  Improvement  Group  (CAIG). 

For  example,  late  in  2013,  the  F-35  Program  Manager  stated  that  the  program  office  O&S  cost  estimate  for  the 
total  force  F-35  cost  (“C”  model)  was  20  percent  less  than  the  ICE  developed  at  the  latest  Nunn-McCurdy 
breach. 

Since  ground  units  contain  a  mix  of  many  kinds  of  equipment,  it  is  more  difficult  to  isolate  the  costs  to  man  and 
support  a  particular  piece  of  equipment  than  for  systems  such  as  aircraft  and  ships. 
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One  approach  to  O&S  affordability  that  bypasses  some  of  those  issues  is  to  compare  the 
estimated  O&S  costs  of  a  new  system  to  the  system  or  systems  that  it  will  replace  (if  the  actual 
costs  are  reliably  known).  This  approach  works  if  there  are  clearly  defined  legacy  or  antecedent 
systems.  For  such  comparisons  to  be  valid  it  is  necessary,  in  so  far  as  possible,  to  define 
consistent  parameters  for  the  systems  being  compared.  The  comparisons  can  be  made  either  on  a 
system-vs-system  basis  or  “force”  basis.  (The  latter  approach  is  relevant,  for  example,  if  the 
number  of  new  systems  in  a  unit  is  different  (usually  lower)  than  for  the  antecedent  systems.  If 
the  new  system  has  higher  O&S  costs  (especially  for  the  unit-cost  comparison),  then  it  should  be 
incumbent  on  the  sponsoring  Component  to  demonstrate  at  milestone  reviews  how  the  higher 
costs  will  be  afforded.  For  example,  affordability  may  be  achieved  by  having  fewer  (but  more 

capable)  units  in  the  force,  deactivation  of  other  units  or  activities  no  longer  needed  when  the 
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new  system  enters  the  force,  or  possibly  lower  operating  rates. 

The  best  comparison  for  assessing  O&S  affordability  is  to  compare  the  future  cost  to 

O'! 

operate  a  portfolio  of  forces.  The  portfolio  should  be  defined  to  include  most,  if  not  all,  the 
forces  into  which  the  new  system  at  issue  will  be  fielded.  This  may  seem  self-evident,  but 
practical  difficulties  arise  that  tend  to  discourage  development  of  such  comparisons.  It  may  be 
necessary  to  make  assumptions  about  the  future  force  structure  and  its  peacetime  deployment 
and  employment  that  may  not  yet  be  fully  determined.  Manning  levels  and  future  training  and 
operating  rates  cannot  be  known  with  any  certainty,  but  must  be  estimated.  Cost  factors  such  as 
the  cost  of  fuel,  other  commodities,  and  manpower  are  also  uncertain.  It  also  must  be  decided 
what  proportion  of  indirect  manpower  and  logistics  costs  to  include.  Sensitivity  to  assumptions 
and  unknown  parameters  should  be  analyzed.  Despite  the  difficulties,  a  forces-based  cost 
comparison  is  the  most  rigorous  way  to  assess  O&S  cost  affordability. 

C.  Selected  Acquisition  Report  (SAR)  O&S  Costs 

Since  1985,  the  SARs  have  been  required  to  have  a  section  on  O&S  costs,  and  the  required 
level  of  detail  and  fidelity  has  increased.  In  the  past,  little  attention  has  been  paid  to  the  O&S 
costs  in  SARs  compared  to  investment  costs.  That  has  changed  with  the  increased  emphasis  on 
O&S  cost  affordability.  The  data  presented  in  this  section  illustrate  some  of  the  problems  and 
issues  discussed  in  the  previous  sections.  In  particular,  the  difficulties  in  making  consistent  and 
reliable  O&S  cost  estimates  is  amply  illustrated.  Furthermore  the  changes  in  estimates  over  time 
illustrate  both  the  uncertainty  and  the  maturation  of  estimates  as  programs  transition  from  early 
to  late  development  to  fielding. 


For  example,  if  a  personnel  training  for  the  new  system  will  make  greater  use  of  simulators. 

If  the  portfolio  is  defined  too  broadly,  the  change  differences  will  appear  less  significant  compared  to  the  cost  to 
operate  the  entire  portfolio.  In  addition  to  the  total  cost,  it  will  frequently  be  useful  to  display  the  cost 
comparison  for  a  unit,  such  as  a  squadron  or  a  brigade. 
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1.  Comparisons  to  Antecedent  Systems 

A  provision  of  the  SAR  that  should  be  of  particular  interest  for  O&S  cost  affordability  is 
the  requirement  (imposed  by  law)  to  compare  the  O&S  costs  of  the  new  system  to  its  antecedent, 
if  one  can  be  clearly  identified.  Both  per-unit  (“unitized”)  cost  and  total  life-cycle  O&S  costs 
estimates  are  required.  The  research  team  obtained  the  results  of  a  query  of  the  SAR  database 
that  provided  all  O&S  cost  information  including  antecedent  system  costs,  when  available.  It  is 
apparent  from  that  data  that  the  Components  have  wide  leeway  in  deciding  whether  to  include  a 
comparison  to  an  antecedent  system — only  forty-two  out  of  some  eighty-three  MDAPs  provided 
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data  on  an  antecedent. 

Figure  4  is  a  comparison  of  unitized  O&S  costs  for  new  systems  versus  antecedent  systems 
for  those  forty-two  systems.  On  average  there  is  a  13  percent  increase  in  O&S  costs  for  new 
compared  to  legacy  systems.  It  is  interesting,  however,  that  fourteen  systems  indicate  lower  O&S 
costs  than  their  antecedents.  These  comparisons,  if  accurate,  provide  a  valuable  way  to  view 
O&S  costs  affordability.  Increases  in  O&S  costs  over  antecedent  systems  are  a  strong  indicator 
of  a  potential  O&S  cost  affordability  problem  in  the  future. 


It  appears  to  be  strictly  up  to  the  Components  to  determine  whether  an  appropriate  antecedent  exists.  For 
example,  the  V-22  Osprey  SAR  states  that  there  is  no  antecedent;  however,  these  aircraft  are  replacing  existing 
rotary  wing  aircraft,  mostly  CH-46  Sea  Knights.  There  appears  to  be  no  reason  why  the  aircraft  being  replaced 
are  not  suitable  as  an  antecedent  for  comparison. 
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Figure  4.  Comparison  of  O&S  Costs  for  New  versus  Antecedent  Systems  (Dec  2012  SARs) 

The  research  team  examined  several  of  these  comparisons  in  more  detail  to  better 
understand  why  some  new  systems  are  projected  to  cost  less  than  antecedents  while  many  others 
cost  more,  some  substantially  more.  One  example  examined  was  the  Army’s  Lakota  Light  Utility 
Helicopter  (LUH),  which  is  a  commercial  helicopter  adapted  for  Army  use  in  supporting  (non¬ 
combat)  roles.  It  was  found  that  the  Army  had  used  as  the  antecedent  the  UH-60L  Black  Hawk. 
While  a  few  Lakotas  (less  than  thirty)  will  replace  UH-60Ls  serving  in  roles  where  their 
capabilities  are  not  needed,  the  vast  majority  of  the  planned  purchase  of  315  Lakotas  will  replace 
OH-58s  and  UH-ls.  Because  the  weight  and  engine  horsepower  of  the  Black  Hawk  is  almost 
three  times  that  of  the  Lakota,  the  OH-5  8 A/C  or  the  UH-1  would  be  a  more  appropriate 
antecedent  for  Lakotas. 

Figure  5  compares  the  O&S  costs  from  the  Dec- 12  SARs  for  several  systems — four  with 
lower  costs  than  the  antecedent  and  four  with  higher.  The  unitized  costs  are  provided  by  the 
categories  of  cost  seen  in  the  SARs.  The  last  entry  for  each  system  is  the  “total  force”  O&S  costs 


U.S.  Army  Homepage,  April  2011,  “Stateside  Lakota  deliveries  let  Black  Hawks  go  to  theater,” 
http://www.armv.mil/article/55394/. 

The  SAR  does  not  provide  a  total  (force)  O&S  cost  for  the  antecedent.  If  it  did,  it  would  be  appropriate  to 
include  the  thirty  UH-60L’s  being  replaced  in  the  antecedent  force  cost. 
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(it  is  not  further  decomposed  in  the  SARs).  While  the  SARs  usually  provide  some  explanatory 
information,  it  varies  greatly  over  systems,  and  in  most  cases,  it  is  not  possible  to  fully 
understand  the  reasons  for  the  differences  from  just  the  information  in  the  SARs,  i.e.,  the  data 
invite  in-depth  analysis  beyond  the  scope  of  this  paper. 

The  following  bullets  discuss  issues  that  emerge  from  the  unitized  cost  comparisons  in 
Figure  5.^^ 

•  Higher  O&S  costs  examples: 

-  KC-130J.  The  increases  in  maintenance  costs  and  sustaining  support  of  over  100 
percent  may  be  attributable  to  the  following  comment  in  the  SAR:  “both  the  KC- 
130F  and  KC-130R  were  Acquisition  Category  II  programs  that  relied  heavily  on 
United  States  Air  Force  (USAF)  program  sustainment.”  This  comment  suggests 
that  not  all  sustainment  costs  for  the  antecedent  systems  are  included.  Also  the  174 
percent  increase  in  continuing  systems  improvement  begs  explanation. 

-  CH-53K.  Since  the  new  K  model  is  significantly  more  capable  than  the 
antecedent — increased  lift  capacity,  operational  range,  and  more  powerful 
engines — it  is  not  surprising  that  support  costs  would  increase.  However,  the 
amount  of  the  increase  could  be  questioned,  including  the  increase  in  “sustaining 
support”  of  over  100  percent.  Regarding  affordability,  there  should  be  concern  that 
the  SAR’s  force  cost  estimate  for  the  CH-53K  is  23  percent  higher  than  for  the  CH- 
53E,  since  it  does  not  appear,  based  on  that  comparison,  that  the  USMC  plans  to 
reduce  the  force  size  for  an  equivalent  force  capability. 

-  JLTV.  Substantial  increases  in  maintenance  and  continuing  systems  improvements 
but  none  in  either  sustaining  support  or  indirect  support  seems  inconsistent.  The 
SAR  only  notes  that  the  antecedent  estimates  are  “rough  Order  Magnitude 

. .  .developed  used  (sic)  JLTV  cost  model  adjusted  with  system  technical  &  cost  data 
for  High-Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV)  (Ml  151,  Ml  152  & 
Ml  165).  HMMWV  data  normalized  for  JLTV  quantity,  operating  schedule, 
OPTLMPO  &  other  Ground  Rules  and  Assumptions.”  It  is  not  known  how  the 
Army  and  Marine  Corps  plan  to  pay  for  this  significant  increase  in  O&S  costs  for 
some  55,000  vehicles. 

-  KC-46.  The  SAR  notes  a  number  of  reasons  to  explain  the  differences  in  O&S  costs 
vs.  the  antecedent  KC-135  aircraft.  The  KC-135  costs  are  based  on  VAMOSC  data 


All  quotes  in  the  bullets  below  are  from  the  notes  to  O&S  Cost  sections  of  the  latest  available  SARs. 

DOD,  Selected  Acquisition  Report  (SAR),  RCS:  DD-A&T(Q&A)823-433,  “KC-130J  Transport  Aircraft  (KC- 
130J)”  (Washington,  DC:  DOD,  December  2012),  35. 

DOD,  SAR  RCS:  DD-A&T(Q&A)823-279,  “Joint  Light  Tactical  Vehicle  (JLTV)”  (Washington,  DC:  DOD, 
December  2012),  49. 
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reflecting  past  costs — the  USAF  expects  future  costs  to  be  higher  as  the  fleet  age 
increases.  Nonetheless,  unless  a  significant  number  of  KC-135s  will  not  be 
replaced,  the  issue  of  how  to  pay  for  a  21  percent  increase  in  O&S  costs  for  the 
tanker  fleet  must  be  addressed.  (The  SAR  does  not  provide  a  total  force  O&S  cost 
for  the  antecedent  fleet — saying  simply:  “A  comparable  total  O&S  cost  for  the 
antecedent  system,  KC-135R&T,  is  not  available” with  no  explanation  of  why 
that  is  the  case.) 

•  Lower  O&S  costs  examples: 

-  JPATS.  The  SAR  provides  no  explanation  for  how  the  decreases  in  unit  manpower 
and  operations  will  be  achieved.  Maintenance  costs  for  the  new  system  are 
projected  to  be  substantially  higher,  while  sustaining  support  and  indirect  support 
are  much  lower.  This  is  indicative  of  a  difference  in  support  concepts.  More 
research  would  be  required  to  understand  whether  this  projected  decrease  in  O&S 
costs  is  realistic.  (Past  experience  suggests  that  projections  of  manpower  reductions 
often  are  not  fully  achieved. 

-  C-130J.  The  1 1  percent  decrease  in  O&S  costs  compared  to  the  antecedent  is  due 
entirely  to  lower  unit  manpower  costs.  The  SAR  offers  no  explanation. 

-  CH-47F.  The  big  decreases  are  in  unit  manpower,  sustaining  support,  and  indirect 
support.  The  latter  two  decrease  much  more  than  maintenance  costs.  The  SAR  notes 
that  “The  CH-47D  and  CH-47F  costs  are  based  on  CH-47D  actuals  extracted  from 
the  Operating  and  Support  Management  Information  System  (OSMIS  [the  Army’s 
VAMOSC]).  To  calculate  the  CH-47F  costs,  these  CH-47D  actuals  were  augmented 
by  an  improvement  factor  to  account  for  the  increased  reliability  of  recapitalized 
parts,  new  airframe,  and  vibration  engineering.”  No  explanation  is  offered  for  the 
decrease  in  unit  manpower,  nor  why  sustaining  support  and  indirect  costs  decrease 
so  much  more  than  maintenance.^^ 

-  EA-18G.  The  significant  increase  in  unit  operations  costs  is  more  than  offset  by 
decreases  in  maintenance,  sustaining  support,  and  continuing  system  improvements. 
While  the  SAR  offers  1 14  pages  of  notes,  how  these  savings  in  support  costs  will  be 


DOD,  SAR  RCS:  DD-A&T(Q&A)823-387,  “KC-46A  Tanker  Modernization  (KC-46A)”  (Washington,  DC: 
DOD,  December  2012),  46. 

As  an  illustration,  the  Dec-01  F-22  SARs  indicate  a  41  percent  reduction  in  manpower  costs  compared  to  the  F- 
15C  but  the  Dec-10  SAR  indicates  27  percent  higher  manpower  costs  for  the  F-22. 

DOD,  SAR  RCS:  DD-A&T(Q&A)823-278  “CH-47F  Improved  Cargo  Helicopter  (CH-47F)”  (Washington,  DC: 
DOD,  December  2012),  34. 

A  research  team  member  discussed  reliability  improvements  in  the  CH-47F  model  over  the  -D  model  with 
members  of  the  CH-47  program  office,  and  was  informed  that  the  improved  reliability  was  attributable  to  the 
fact  that  the  -F  models  are  newer  and  not  to  any  explicit  efforts  to  improve  reliability. 
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achieved  is  not  explained.  The  Navy  does  not  intend  to  replaee  all  its  EA-6Bs  with 
EA-lSGs. 


Higher  O&S  Costs 

Lower  O&S  Costs 

Budget  Year  2010 

KC-130J 

KC-130  F/R/T 
(Antecedent) 

Percent 

Change 

Budget  Year  2002 

JPATS 

T-37{AFOnly) 

(Antecedent) 

Percent 

Change 

Average  Annual  Cost  per  Aircraft 

Average  Annual  Per  Aircraft 

Unit-Level  Manpotwer 

2283.314 

2283.314 

0% 

Unit  Level  Manpower 

0.180 

0.260 

-31% 

Unit  Operations 

1678.787 

1360.657 

23% 

Unit  Operations 

0.052 

0.310 

-83% 

Maintenance 

4227.593 

1878.000 

125% 

Maintenance 

0.095 

0.020 

375% 

Sustaining  Support 

341.235 

131.000 

160% 

Sustaining  Support 

0.062 

0.110 

-44% 

Continuing  System  Improvements 

868.347 

317.000 

174% 

Conintuing  System  Improvements 

0.000 

0.000 

Indirect  Support 

488.769 

488.769 

0% 

Indirect  Support 

0.040 

0.200 

-80% 

Other 

0.000 

0.000 

other 

0.000 

0.000 

Total 

9888.045 

6458.740 

53% 

Total 

0.429 

0.900 

-52% 

Force  Total 

39987.3 

26119.1 

53% 

Force  Total 

11697.8 

"NA" 

Budget  Year  2006 

CH-53K 

CH-53E 

(Antecedent) 

Percent 

Change 

Budget  Year  1996 $1000 

C-130J 

C-130H1  &  H2 
(Antecedent) 

Percent 

Change 

Average  Annual  Cost  per  Helicopter 

Avg  Annual  Cost  per  Aircraft 

Unit-Level  Manpower 

1165.200 

1239.700 

-6% 

Unit-Level  Manpower 

1754.200 

2314.900 

-24% 

Unit  Operations 

360.300 

308.800 

17% 

Unit  Operations 

613.700 

490.900 

25% 

Maintenance 

4789.800 

3247.400 

47% 

Maintenance 

1280.400 

1309.700 

-2% 

Sustaining  Support 

221.700 

106.400 

108% 

Sustaining  Support 

39.300 

39.300 

0% 

Continuing  System  Improvements 

552.000 

703.600 

-22% 

Continuing  System  Improvements 

44.300 

44.300 

0% 

Indirect  Support 

498.200 

564.400 

-12% 

Indirect  Support 

234.300 

234.300 

0% 

Other 

0.000 

0.000 

Other 

0.000 

0.000 

Total 

7587.200 

6170.300 

23% 

Total 

3966.200 

4433.400 

-11% 

Force  Total 

37496.2 

30494.1 

23% 

Force  Total 

19989.6 

22344.5 

-11% 

Budget  Year  2012 

JLTV 

HMMWV 

(Antecedent) 

Percent 

Change 

Budget  Year  2005 

CH-47F 

CH-47D 

(Antecedent) 

Percent 

Change 

Average  Annual  Cost  per  Vehicle 

A  verage  Annual  per  A  ircraft 

Unit-Level  Manpower 

8.700 

8.700 

0% 

Unit  Level  Manpower 

409.484 

658.828 

-38% 

Unit  Operations 

5.300 

5.800 

-9% 

Unit  Operations 

70.107 

76.408 

-8% 

Maintenance 

12.200 

7.100 

72% 

Maintenance 

1200.573 

1208.797 

-1% 

Sustaining  Support 

1.200 

1.200 

0% 

Sustaining  Support 

16.944 

470.291 

-96% 

Continuing  System  Improvements 

1.700 

0.800 

113% 

Continuing  System  Improvements 

127.061 

11.359 

1019% 

Indirect  Support 

0.000 

0.000 

Indirect  Support 

101.131 

652.265 

-84% 

other 

0.000 

0.000 

other 

0.000 

0.000 

Total 

29.100 

23.600 

23% 

Total 

1925.300 

3077.948 

-37% 

Force  Total 

31708.4 

25800.9 

23% 

Force  Total 

16942.6 

18837 

-10% 

Budget  Year  2011 

KC-46A 

KC-135R&T 

(Antecedent) 

Percent 

Change 

Budget  Year  2004 

EA-18G 

"Antecedent" 

EA-6B 

(Antecedent) 

Percent 

Change 

Average  Annual  Cost  per  Aircraft 

A  verage  Annual  Cost  per  A  ircraft 

Unit-Level  Manpower 

4.400 

3.200 

38% 

Unit  Level  Manpower 

2.435 

2.228 

9% 

Unit  Operations 

4.000 

4.100 

-2% 

Unit  Operations 

0.780 

0.550 

42% 

Maintenance 

3.200 

3.100 

3% 

Maintenance 

2.935 

3.313 

-11% 

Sustaining  Support 

0.500 

0.200 

150% 

Sustaining  Support 

0.183 

0.363 

-50% 

Continuing  System  Improvements 

0.900 

0.100 

800% 

Continuing  System  Improvements 

0.941 

1.603 

-41% 

Indirect  Support 

0.000 

0.000 

Indirect  Support 

0.518 

0.448 

16% 

other 

0.000 

0.000 

other 

0.000 

0.000 

Total 

13.000 

10.700 

21% 

Total 

7.792 

8.505 

-8% 

Force  Total 

103090.5 

"NA" 

Force  Total 

16284.0 

17797 

-9% 

Figure  5.  Comparisons  of  Unitized  and  Force  O&S  Cost  Estimates  for  Seiected  Systems  ($1,000) 
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For  assessing  O&S  affordability,  the  most  desirable  eomparison  is  between  the  foree  with 
the  new  system  and  the  foree  without  the  new  system.  Unfortunately,  that  eomparison  is  more 
diffieult  sinee  more  assumptions  are  required.  For  a  number  of  SARs  reviewed  (not  just  those  in 
the  table),  it  does  not  appear  that  the  “Total  O&S  Cost”  table  provides  a  eomparison  valid  for 
that  purpose.  For  example,  for  the  CVN  78  nuelear  earrier,  the  Navy  seems  to  eompare  a  single 
CVN  78  to  the  entire  foree  of  the  anteeedent  (identified  as  the  CVN  68).  In  the  MQ-9  Reaper 
SAR,  the  total  eost  for  the  404  MQ-9s  being  proeured  is  eompared  to  the  mueh  smaller  foree  of 
MQ-1  Predators,  thus  showing  an  inerease  in  O&S  costs  of  $32  billion  ($37.9  billion  versus  $5.7 
billion)  over  the  antecedent  (whereas  the  cost  per  flying  hour  of  the  Reaper  is  only  about  6 
percent  higher  than  the  Predator).  That  comparison  raises  the  issue  of  how  the  larger  force  will 
be  afforded,  but  that  is  more  of  a  force  size  issue  than  an  acquisition  issue. 

Another  example  is  the  DDG  5 1 ;  The  antecedent  identified  in  the  SAR  is  the  CG  47  cruiser, 
and  the  DDG  51  is  shown  to  have  16  percent  lower  unitized  costs.  However,  the  total  force  cost 
comparison  shows  a  $58  billion  higher  cost  (or  158  percent)  for  the  DDG  51  force!  The  notes 
state  that  this  is  for  a  force  of  seventy-seven  DDG  51s,  but  the  number  of  the  ships  for  the 
antecedent  system  is  not  provided.  A  valid  assessment  of  force  O&S  cost  affordability  would 
require  a  resolution  of  the  types  of  problems  discussed  in  this  paragraph.  This  and  similar 
problems  illustrate  issues  discussed  at  the  end  of  the  previous  section. 

2,  Trends  in  O&S  Costs  Reflected  in  SARs 

SAR  data  viewed  over  time  provide  insights  into  the  changes  in  O&S  cost  estimates  as 
programs  mature — generally,  like  investment  costs,  they  show  increases,  sometimes  dramatic. 
Figure  6  shows  O&S  cost  estimate  trends  for  several  high-interest  programs,  displaying  both  the 
new  system  and  the  antecedent  system. 
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Figure  6.  Trends  in  SAR  Unitized  O&S  Costs  for  Seiected  Systems  (Dec.  SARs) 

While  explanatory  information  in  the  SARs  is  generally  too  sparse  to  understand  fully  what 
drove  these  changes,  the  research  team  extracted  available  insights  to  produce  the  notes  below. 
The  increases  in  O&S  costs  of  the  antecedent  systems  presumably  reflect  increases  in  actual 
costs  for  deployed  systems. 

•  F-35  (CTOL).^^  There  was  a  57  percent  increase  for  F-35  and  14  percent  for  the 

antecedent  F-16C  in  unitized  O&S  costs  from  the  Dec-05 to  Dec-06  SARs.  The 
largest  increase  for  the  F-35  was  in  “sustaining  support,”  whereas  for  the  F-16C  the 
largest  increase  was  in  depot  maintenance.  No  explanation  for  these  changes  is  offered 
in  the  Dec-06  SAR.  The  large  increases  in  F-35C  costs  in  2012^^  reflect  a  new 
independent  cost  estimate  by  CAPE  associated  with  a  Nunn-McCurdy  breach.  There  is 
also  a  67  percent  increase  in  the  F-16C  figure  in  the  Dec-1 1  SAR  and  another  1 1 
percent  increase  in  the  Dec- 12  SAR;  neither  was  explained. 


Estimates  are  for  the  CTOL  variant;  the  SAR  provides  no  estimates  for  other  variants. 

The  Dec-05  SAR  O&S  costs  are  identical  to  the  Dec-04  SAR  costs  shown  in  the  graph. 

The  increase  was  actually  first  seen  in  the  Dec-1 1  SAR;  the  Dec- 12  SAR  figures  were  the  same  as  the  Dec-1 1 
for  the  F-35,  but  the  F-16C  figure  increased  by  1 1  percent. 
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•  CH-47F.  The  research  team  was  unable  to  understand  the  SAR  CH-47F  and  CH-47D 
O&S  costs  based  on  information  in  the  SARs.  The  CH-47Ds  have  been  in  service  since 
the  1980s  (the  last  procurement  was  in  2002),  and  the  F  models  have  been  in  service 
since  2002,  so  presumably  a  lot  of  data  on  operating  costs  should  be  available  from 
OSMIS.  Instead  of  using  OSMIS  data  for  the  CH-47F,  the  Army  bases  the  O&S  costs  in 
the  SARs  on  scaling  from  the  CH-47D  values  (said  to  be  from  OSMIS): 

To  calculate  the  CH-47F  costs,  these  CH-47D  actuals  were  augmented  by  an 
improvement  factor  to  account  for  the  increased  reliability  of  recapitalized  parts,  a 
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new  airframe,  and  vibration  engineering. 

OSMIS  data  on  the  two  systems  was  obtained  by  the  research  team  and,  while  they  could 
not  be  tracked  to  the  data  in  the  SAR,  they  do  indicate  lower  operating  costs  for  the  F-model  for 
the  few  items  with  comparable  data,  but  not  the  37  percent  lower  figure  seen  in  the  SAR.  Why 
the  Army  does  not  use  actual  data  from  CH-47F  operations  for  the  SAR  estimate  is  not  known. 

•  F-22.  The  Dec-97  (earliest  available  in  Defense  Acquisition  Management  Information 
Retrieval  System  (DAMIR)  through  the  Dec-OI  SARs,  state  that  the  antecedent  F-I5C 
O&S  costs  are  based  on  VAMOSC,  whereas  the  F-22  costs  are  based  on  “. . .  a 
combination  of  AFI  65-503  Cost  and  Planning  Factors  and  information  provided  in  the 
contractor’s  Affordability  Analysis. There  is  no  statement  regarding  comparability  of 
the  two  systems. The  Dec-02  SAR,  however,  stated: 

In  December  2000,  the  Air  Force  Cost  Analysis  Improvement  Group  (AFCAIG) 
worked  with  the  F/A-22  System  Program  Office  and  the  F-15  System  Program 
Office  to  develop  updated  estimates  of  both  the  F/A-22  and  F-15C  to  provide  an 
equitable  comparison  of  ownership  costs. 


Despite  this  apparent  change  there  was  absolutely  no  change  in  the  unitized  O&S  costs 
between  the  Dec-01  and  Dec-02  SARs.  In  Dec-05  however,  the  O&S  costs  for  the  F-22 
increased  by  41  percent  over  the  previous  year,  whereas  the  F-I5C  costs  decreased  by  4 
percent.  There  is  no  explanation  other  than  changing  the  date  of  the  referenced 
AFCIAG  study  from  2000  to  2004.  By  then  ninety  F-22s  had  been  delivered  so  perhaps 
the  increase  reflects  actual  costs  (though  this  was  not  stated  in  the  SAR). 


Dec- 12  SAR,  33. 

The  data  are  not  displayed  because  all  VAMOSC  data  is  labeled  For  Official  Use  Only. 

DOD,  SAR  RCS:  DD-A&T(Q&A)823-265,  “F-22  Raptor  Advanced  Tactical  Fighter  Aircraft  (F-22)” 
(Washington,  DC,  DOD:  December  2001),  42. 

Certainly  one  would  think  that  the  dramatically  lower  O&S  costs  for  the  much  more  advanced,  stealthy  F-22 
compared  to  the  F-15C  would  have  raised  questions. 

DOD,  SAR  RCS:  DD-A&T(Q&A)823-265F-22,  “Raptor  Advanced  Tactical  Fighter  Aircraft  (F-22)” 
(Washington,  DC:  DOD,  December  2002),  48. 
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•  F/A-18E/F.  In  the  Dec-04  SAR,  the  F/A-18E/F  O&S  costs  increased  43  percent  over 

the  previous  year  while  the  antecedent  F/A-18C/D  increased  35  percent,  partially 
explained  by  a  change  in  the  assumed  price  of  fuel.  In  the  Dec-06  SAR,  F/A-18F/F 

costs  decreased  by  17  percent  over  the  previous  year,  with  no  change  for  the  antecedent. 

1 02 

Neither  are  there  any  explanations  for  the  changes  from  the  Dec-07  to  Dec-09  SAR, 
or  to  the  Dec- 10  SAR. 

The  research  team  obtained  an  extract  of  O&S  cost  data  for  all  SAR  programs  from  the 
DAMIR  database  with  the  intention  of  computing  statistics  on  O&S  cost  growth  for  the  entire  set 
of  MDAPs.  Unfortunately  extensive  inconsistencies  in  the  data  made  that  objective  unachievable 
within  the  available  resources.  However,  four  additional  programs  examined  provide  some 
points  of  interest  (Figure  7).  In  these  cases,  no  comparison  was  made  with  antecedent  programs 
(for  three  of  the  four  no  antecedents  were  named  in  the  SARs),  but  the  trends  in  the  O&S  cost 
estimates  were  developed  (and  compared  to  the  trends  in  unit  procurement  costs).  Two 
characteristics  are  observed:  (1)  O&S  costs  tend  to  increase,  sometimes  significantly,  from  the 
Milestone  II/Milestone  B  to  the  Milestone  Ill/Milestone  C  estimates;  (2)  trends  in  SAR  O&S 
cost  estimates  do  not  appear  to  correlate  strongly  to  procurement  unit  cost  trends,  as  might  be 
expected.  (Obviously  the  sample  size  is  far  too  small  for  statistical  analysis.)  Another  tentative 
finding  is  that,  in  many  if  not  most  cases,  SAR  O&S  costs  in  the  past  do  not  appear  to  have  been 
updated  on  an  annual  basis,  between  the  decision  to  enter  full-scale  development  and  the 
decision  for  full-scale  production.  These  indications  have  implications  for  O&S  affordability 
goals  and  caps.  To  be  enforceable  these  controls  must  be  based  on  realistic  estimates  (else  the 
decision  to  proceed  with  the  program  based  on  its  affordability  will  be  faulty)  and  O&S  costs 
over  time  must  be  re-examined  at  least  annually  to  prevent  a  big  “surprise”  increase  in  O&S 
costs  at  the  decision  point  to  enter  production  and  eventually  field  the  system. 
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There  were  no  Dec -08  SARs. 
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FY2004  $K  FY  2005  $M 


Figure  7.  Trends  in  SAR  O&S  Costs  (Biue  Line)  Compared  with  Procurement  Costs  (Red  Line)  for 

Seiected  MDAPs 
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4.  Controlling  0«&S  Costs  within  Affordability 

Constraints 


This  chapter  will  discuss  approaches  to  managing  programs  so  that  O&S  affordability 
eonstraints  are  not  exeeeded.  Topies  inelude  the  need  to  make  systematie  (and  possibly  periodie) 
tradeoffs  between  eost  (investment  and  O&S)  and  other  program  oharaeteristies  sueh  as  teehnieal 
and  operational  performance,  timing,  procurement  quantities,  and  sehedule;  and  eontrolling  O&S 
eosts  through  the  simpler  approaeh  of  eontrolling  O&S  eost  drivers.  The  potential  benefits  of  a 
standard  O&S  eost  model  implemented  early  and  updated  often  are  outlined.  Lastly  it  also 
discusses  ways  to  better  motivate  both  DOD  and  industry  to  manage  programs  that  remain  within 
O&S  eost  goals  and  how  proeesses  to  aehieve  that  end  ean  be  better  institutionalized. 

A.  Monitoring  Goals  and  Caps 

Effeetively  monitoring  affordability  goals  and  eaps  is  neeessary  if  program  affordability  is 
going  to  be  aehieved.  The  intent  of  BBP  2.0  and  the  newly  published  DODI  5000.02  is  to  use 
goals  and  eaps  to  ensure  that  programs  eontinue  to  be  affordable  as  they  progress  through  the 
aequisition  proeess.  Expeetations  are,  therefore,  that  at  Milestone  A  and  Milestone  B,  DOD 
Components  will  either  (1)  have  eost  estimates  that  are  within  the  assoeiated  goal  or  eap  or  (2) 
be  able  to  provide  eonvineing  rationale  why  the  program  will  be  able  to  lower  the  costs  to  meet 
the  goal  or  eap.  This  latter  situation  is  more  likely  to  oecur  at  Milestone  A  than  Milestone  B  for 
O&S  eosts  sinee  many  important  deeisions  driving  future  eost  will  be  made  during  the  TMRR 
phase. 

In  most  oases,  oost-sohedule-performanoe  trade  studies  only  address  investment  eosts. 
While  designers  may  oonsider  how  a  design  deoision  will  affect  investment  eost,  they  may  not 
have  effective  tools  for  making  sueh  tradeoffs  because  many  O&S  eost  implications  of  design 
deeisions  may  not  always  be  well  understood.  Sinee  reliability  and/or  maintainability  are 
normally  addressed  in  either  KPPs  or  Key  System  Attributes  (KSA),  reliability  and 
maintainability  engineers  would  normally  work  olosely  with  designers  to  faoilitate  design  trades. 
However,  serious  tradeoffs  are  not  likely  to  happen  unless  meeting  an  O&S  eost  target  has  been 
defined  as  a  firm  eonstraint.  Based  on  the  researeh  team’s  discussions  with  several  industry 
subject  matter  experts,  staffing  priority  may  not  be  given  to  ineluding  reliability  and 
maintainability  engineers  early  enough  to  inform  trade  studies  unless  the  eontraet  speeifieally 


These  discussions  were  agreed  to  strictly  on  a  non-attribution  basis. 
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requires  them.  In  praetice  it  is  mueh  more  common  to  make  trades  to  solve  technical  problems 
and/or  reduce  acquisition  cost  that,  in  fact,  increase  O&S  cost. 

In  addition,  despite  the  improvement  in  O&S  cost  estimation  methodologies  since  the  time 
of  DTC  and  CAIV,  there  is  wide  agreement  that  the  level  of  fidelity  that  can  be  expected  for 
O&S  cost  models  for  most  defense  systems  is  still  below  that  for  modeling  production  costs. 
When  cost  estimates  are  based  on  analogy, weaknesses  in  VAMOSC  data  and  known 
differences  between  the  new  and  antecedent  systems  may  lead  to  biases  that  are  difficult  to 
mitigate.  The  fidelity  of  early  O&S  cost  estimates  is  not  sufficient  to  produce  high  confidence 
that  the  actual  O&S  costs  of  the  system  when  fielded  will  be  below  the  cap.  Force  structure  and 
operating  rates  different  from  those  assumed  for  the  estimates  would  not  be  unusual. 
Furthermore,  post-Milestone  B  O&S  cost  estimates  have  not  always  been  updated  regularly  in 
the  SARs.'°^ 

Experience  has  shown  that  O&S  costs  for  fielded  systems  (even  when  adjusted  for  external 
factors  such  as  OPTEMPO  rates  and  cost  of  fuel)  often  turn  out  to  be  higher  than  estimated 
during  development.  Reliability  and  maintainability  will  be  better  understood  after  operational 
testing  and  initial  fielding.  Manning  and  product  support  strategies  will  be  finalized.  OPTEMPO 
will  be  better  defined.  Unfortunately,  most  of  these  data  only  become  available  after  it  is  too  late 
(or  too  costly)  to  make  design  changes  in  the  basic  system. 

O&S  cost  models  are  in  general  use  in  both  DOD  and  industry;  however,  there  is  no 
requirement  in  DODI  5000.02  to  establish  and  use  such  a  model  to  help  manage  O&S  costs  for  a 
system  under  development.  To  do  so  within  a  standard  modeling  framework  could  greatly 
facilitate  tracking  and  managing  O&S  costs.  Initially,  such  a  model  could  be  developed  from 
cape’s  independent  cost  estimate  for  the  Milestone  A  decision.  The  model  framework  could  be 
agreed  upon  by  all  stakeholders  and  its  use  required  of  the  TMRR  contractors.  The  model  would 
be  continuously  refined  and  updated,  as  designs  mature  approaching  Milestone  B,  characteristics 
become  better  defined,  and  engineering-based  O&S  cost  estimates  developed.  The  process  would 
continue  post-Milestone  B,  and  eventually  when  preliminary  test  results  become  available  to 
validate  the  model. 

Between  Milestones  A  and  B,  monitoring  evolving  O&S  costs  vis-a-vis  the  affordability 
goal  may  be  more  problematic.  Up  to  the  point  of  the  pre-Milestone  B  Preliminary  Design 
Review,  information  on  O&S  costs  will  most  likely  reside  with  the  contractor,  and  there  is 
currently  no  requirement  to  report  such  data.  The  existence  of  a  standard  agreed-to  O&S  cost 


For  example,  early  in  the  conceptual  stage  before  more  detailed  cost  estimating  relationships  are  developed. 

See  GAO,  DOD  Needs  Better  Information  and  Guidance  to  More  Effectively  Manage  and  Reduce  Operating 
and  Support  Costs  of  Major  Weapon  Systems,  GAO-10-717  (Washington,  DC:  GAO,  July  2010). 

Evidence  is  provided  by  the  SAR  data  displayed  in  Chapter  3,  which  shows  increases  in  O&S  costs  estimates 
after  system  fielding. 
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modeling  framework,  however,  should  faeilitate  traeking  and  support  tradeoff  analyses  that 
address  O&S  eosts. 

Post-Milestone  B,  the  eurrent  plan  is  to  monitor  affordability  eaps  after  Milestone  B  within 
the  DAES  proeess.  The  following  bullets  points  deseribe  the  value  of  sueh  improved  monitoring 
activities. 

•  The  modeling  framework  described  above  would  be  particularly  valuable  because  the 
assumptions  behind  all  O&S  cost  estimates  beginning  with  Milestone  A  would,  of 
necessity,  be  fully  documented  and  agreed  to. 

•  Annual  updates  of  O&S  costs  in  SARs  should  be  the  norm,  and  would  flow  readily 
from  the  O&S  cost  model,  if  it  exists. 

An  ancillary  point  is  that,  in  order  to  assess  O&S  cost  affordability  within  a  portfolio 
framework  as  required  by  DODI  5000.02,  O&S  costs  for  all  systems  in  the  portfolio  would  have 
to  be  estimated  consistently. 

B.  O&S  Cost  Tradeoffs 

After  approval  at  MDD,  a  new  acquisition  program  (and  presumed  future  MDAP,  or  “pre- 
MDAP”)  enters  the  Material  Solutions  Analysis  phase  of  the  acquisition  process.  During  this 
phase,  the  AoA  will  be  conducted,  including  initial  trade  studies  between  cost,  schedule,  and 
performance — in  fact,  virtually  all  aspects  of  the  program  should  be  subject  to  tradeoffs.  DODI 
5000.02  has  the  following  guidance: 

The  analysis  will  . . .  focus  on  identification  and  analysis  of  alternatives;  measures 
of  effectiveness;  key  trades  between  cost  and  capability;  total  life  cycle  cost, 
including  sustainment;  schedule;  concepts  of  operations;  and  overall  risk.  The 
AoA  will  inform  and  be  informed  by  affordability  analysis,  cost  analysis, 
sustainment  considerations,  early  systems  engineering  analyses,  threat 

1  07 

projections,  and  market  research. 

This  guidance  is  quite  clear  and  specific  that  O&S  costs  should  be  included  in  the  studies. 
However,  the  Instruction  does  not  specify  that  a  Department  level  oversight  body  be  established 
to  oversee  the  AoA  and  related  trade  studies.  There  is  only  a  requirement  that  DCAPE  assess  the 
AoA  after  its  completion  in  a  memorandum  to  the  DAE,  including  whether  it  adequately 
“Considers  tradeoffs  among  cost,  schedule,  sustainment,  and  required  capabilities.”  Assuming 
that  DCAPE  scores  the  AoA  as  adequate  in  that  regard,  it  can  be  assumed  that  at  Milestone  A  the 
systems,  if  developed  successfully  as  planned,  will  be  affordable  from  both  an  investment  and 
O&S  costs  perspectives. 
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DODI  5000.02,16. 
Ibid.,  123. 
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In  the  next  phase  (TMRR),  the  design  concept  will  be  translated  into  actual  hardware — 
usually  in  a  competitive  development  program.  At  this  point,  the  major  responsibility  for 
ensuring  that  the  materiel  solutions  remain  within  affordability  goals  will  lie  with  the  contractors. 
There  is  no  responsibility  for  the  contractors  to  work  with  or  report  to  the  government  on 
whatever  tradeoffs  they  find  it  necessary  to  make,  though  it  might  be  hoped  that  PMs  will  be 
informed  of  such  decisions  informally.  (The  considerations  that  will  likely  guide  such  contractor 
studies  will  be  addressed  in  the  next  section  of  this  paper.) 

At  the  completion  of  the  TMRR  contracts,  a  request  for  proposals  (RFP)  for  the  EMD  phase 
will  be  issued.  A  critical  decision  point  for  the  DAB  is  the  approval  of  that  RFP.  The  DODI 
5000.02  identifies  this  decision  as  the  “Development  RFP  Release  Decision  Point:” 

This  review  is  the  critical  decision  point  in  an  acquisition  program.  The  program 
will  either  successfully  lead  to  a  fielded  capability  or  fail,  based  on  the  soundness 
of  the  capability  requirements,  the  affordability  of  the  program,  and  the 
executability  of  the  acquisition  strategy.  The  acquisition  strategy  is  put  into 
execution  at  this  decision  point  by  asking  industry  for  bids  that  comply  with  the 
strategy.  Release  of  the  RFP  for  EMD  sets  in  motion  all  that  will  follow.  This  is 
the  last  point  at  which  significant  changes  can  be  made  without  a  major 
disruption. 

Thus  the  designs  selected  for  full-scale  development  must  have  the  credible  potential  to 
provide  the  necessary  capabilities  within  affordable  life-cycle  costs.  Since  the  most  likely 
responders  to  the  REP  are  the  TMRR  contractors,  their  responses  will  be  based  largely  on  the 
designs  already  developed  in  that  phase.  If  the  TMRR  contractors  have  adequately  considered 
life-cycle  affordability  constraints  in  their  trade  studies,  they  will  be  able  to  offer  designs  for 
EMD  that  will  be  affordable.  In  turn  that  will  have  happened  if  the  TMRR  REP  and  contract 
under  which  that  development  proceeded  appropriately  stressed  life-cycle  affordability. 

Prior  to  Milestone  B,  the  Joint  Staff  will  validate  the  Component’s  Capability  Development 
Document  (CDD),  which  refines  the  specification  of  the  requirements  for  the  system  for  the 
EMD  phase.  At  that  point,  a  Configuration  Steering  Board  (CSB)  is  constituted  by  the 
sponsoring  Component’s  Acquisition  Executive  with  members  from  all  the  key  stakeholders, 
including  OSD  and  Joint  Staff  This  body,  which  is  required  by  law,^''  is  charged  to  “review 
potential  capability  requirements  changes  and  propose  to  the  requirements  validation  authority 
those  changes  that  may  be  necessary  to  achieve  affordability  constraints  on  production  and 


Ibid.,  21-22. 

DODI  5000.02  states  that  “the  Acquisition  Executive  of  each  DoD  Component  will  form  and  chair  a  CSB  with 
broad  executive  membership  including  senior  representatives  from  the  Office  of  the  USD(AT&L)  (including 
the  Assistant  Secretary  of  Defense  for  Acquisition),  the  Joint  Staff  (DJ8),  and  the  DoD  CIO;  empowered 
representatives  from  the  Service  Chief  of  Staff  and  comptroller  offices  of  the  Military  Department  concerned; 
representatives  from  other  Military  Departments  where  appropriate;  the  Military  Deputy  to  the  CAE;  the  PEO; 
and  other  senior  representatives  from  OSD  and  the  DoD  Component.”  DODI  5000.02,  21. 

This  requirement  is  found  in  Section  814  of  Public  Law  (P.L.)  1 10-417. 
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sustainment  costs  or  that  will  result  in  a  more  cost-effective  product.”  According  to  a  diagram 
in  DODI  5000.02,  CDD  validation  will  occur  just  prior  to  the  decision  point  approving  the  EMD 
RFP.  Thus  key  cost-performance  trades  made  before  EMD  REP  release  could  be  made  under 
the  auspices  of  the  CSB;  however,  it  is  not  clear  that  there  will  be  adequate  time  for  such  a 
complex  activity,  including  coordination  with  the  relevant  stakeholders,  to  take  place  in  time  to 
affect  the  REP.  An  alternative  would  be  to  constitute  the  CSB  earlier  than  currently  required  by 
law  and  regulation — at  least  by  Milestone  A  and  possibly  sooner. 

If  the  process  is  operating  as  designed,  at  the  start  of  EMD  the  design  or  designs  will  be 
compatible  with  the  affordability  cap.  However,  it  is  not  at  all  unusual  for  difficulties  to  occur 
soon  after  Milestone  B  that  will  require  additional  cost-schedule-performance  tradeoffs.  The 
CSB  is  the  body  responsible  for  overseeing  trade  studies. 

O&S  costs  should  figure  prominently  in  these  trade  studies;  however,  as  a  practical  matter 
O&S  costs  will  usually  be  uncertain  prior  to  Milestone  B  and  even  well  thereafter.  That 
likelihood  argues  for  a  management  focus  on  the  drivers  of  O&S  costs,  which  are  subject  to 
control  throughout  the  development  process.  The  next  section  focuses  on  that  approach. 

C.  Managing  O&S  Cost  Drivers 

IDA  has  performed  analyses  of  actual  annual  O&S  costs  for  Air  Eorce  and  Navy  fighters 
from  1997  to  2013  for  other  research  efforts.  It  was  found  that  unit-level  manpower,  unit 
operations  (primarily  fuel  consumption),  and  maintenance  (which  depend  on  system  reliability 
and  maintainability)  account  for  --SS  percent  of  the  system’s  O&S  costs.  These  figures  break  out 
as  follows: 


Table  4.  Fraction  of  O&S  Costs  Determined  by  Unit  Manning, 
Unit  Operating,  and  Maintenance  Costs 


Air  Force 

Navy 

Unit-level  manpower 

29% 

22% 

Unit  operations 

27% 

28% 

Maintenance 

32% 

38% 

Total  Above 

88% 

88% 

Erom  that  analysis  it  appears  that,  for  aircraft,  the  drivers  for  O&S  cost  are  manpower,  fuel, 
and  maintenance  costs.  While  in  many  cases  these  costs  are  direct  consequences  of  system 
requirements,  the  acquisition  system  does  have  some  ability  to  control  them.  Working  with  both 
the  industry  team  and  the  requirements  community,  the  program  office  can  control  reliability, 
maintainability,  and  fuel  consumption  rates  to  some  extent.  Similarly,  the  program  office  can 


DODI  5000.02,21. 

DODI  5000.02.  9,  Figure  3. 
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ensure  that  the  produet  support  strategy  is  enabled  by  industry  design  teams.  However,  key 
external  factors  including  operational  manning,  fuel  cost,  inflation  rate  assumptions,  and 
operating  tempo  are  not  determined  by  the  supportability  of  the  system  and  are  outside  the 
control  of  the  acquisition  process. 

A  starting  point  for  managing  cost  drivers  is  managing  the  framing  assumptions  behind  the 
program.  For  O&S  cost  estimation,  framing  assumptions  will  normally  relate  strongly  to  the 
O&S  cost  drivers.  For  example,  if  a  framing  assumption  that  there  will  only  be  two  levels  of 
maintenance  (organizational  and  depot)  for  a  system  were  to  change  to  also  include  intermediate- 
level  maintenance,  then  there  might  be  significant  changes  to  the  cost  estimate. 

Reliability  and  maintainability  (R&M)  have  long  been  recognized  as  important  drivers  of 
O&S  costs.  Despite  that  recognition,  in  the  zeal  for  acquisition  reform  in  the  past,  specific  R&M 
objectives  were  sometimes  not  included  in  system  KPP  or  KSAs  for  MDAPs.  Serious  reliability 
issues  in  programs  such  as  JASSM  and  SBIRS  (discussed  in  Appendix  A)  resulted  in  a  re- 
evaluation  of  that  policy  and  the  subsequent  inclusion  of  more  stringent  “sustainment  KPPs”  for 
acquisition  programs.  Even  though  the  interim  DODI  5000.02  provides  significant  guidance  on 
how  R&M  is  to  be  managed  in  the  acquisition  process,  a  requirement  to  include  R&M  in  one  or 
more  KPPs  or  KSAs,  though  implied,  is  not  stated  explicitly.  The  Instruction  requires  a 
“Reliability,  Availability,  Maintainability,  and  Cost  Rationale  (RAM-C)  Report  in  support  of  the 
Milestone  A  decision  to  provide  “a  quantitative  basis  for  reliability  requirements,  and  improves 
cost  estimates  and  program  planning.”"^  The  report  is  updated  prior  to  the  pre-Milestone  B 
development  RFP  release  decision  point  and  is  an  attachment  to  the  program’s  Systems 
Engineering  Plan. 

Thus,  managing  the  O&S  cost  drivers  themselves  may  be  the  most  feasible  approach  to 
controlling  O&S  costs.  Even  though  it  may  not  be  possible  to  precisely  estimate  the  O&S  cost 
associated  with  particular  cost  drivers,  efforts  to  attack  those  drivers  will  likely  have  a  high 
payoff  in  reducing  O&S  costs.  As  an  example,  previous  IDA  research  considered  the  C-I7  airlift 
aircraft,  and  determined  that  seven  cost  drivers  collectively  determined  roughly  75  percent  of  the 
system  O&S  costs.  In  general,  insights  can  be  derived  from  experiences  with  legacy  systems. 
Reliability  can  be  increased,  improved  built-in  tests  can  be  implemented,  and  improved  access  to 
components  for  maintenance  can  be  enhanced. 

Based  on  all  of  these  observations,  it  can  be  concluded  that 


DODI  5000.02  references  the  “Manual  for  the  Operation  of  the  Joint  Capabilities  Integration  and  Development 
System;”  however,  that  document  is  advisory  and  is  not  published  under  the  signature  of  an  authority.  The  Joint 
Staff  Instruction  CJCS  3170.01H,  “JOINT  CAPABILITIES  INTEGRATION  AND  DEVELOPMENT 
SYSTEM,”  is  signed  by  the  Director  of  the  Joint  Staff  but  it  does  not  discuss  R&M  requirements  (but  does 
reference  the  Manual). 

DODI  5000.02,  83-84. 

Harold  S.  Balaban,  Waynard  C.  Devers,  and  Lance  M.  Roark,  Feasibility  and  Advisability  of  Baselines  for  O&S 
Costs:  C-17  Case  Study,  IDA  Document  NS  D-4088  (Alexandria,  VA:  IDA,  August  2010). 
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•  Although  quantitative  impacts  of  managing  controllable  O&S  cost  drivers  will  be 
uncertain,  activities  to  manage  them  will  likely  have  a  high  payoff. 

•  Identification  and  monitoring  of  O&S-specific  framing  assumptions  will  facilitate 
managing  O&S  cost  drivers. 

•  A  standard  O&S  cost  model  established  at  or  before  Milestone  A,  as  discussed  above, 
should  greatly  facilitate  tracking  progress  in  controlling  O&S  costs. 

These  steps  would  decrease  the  risk  that  O&S  costs  will  exceed  goals  or  caps. 

D.  Incentives  to  Reduce  O&S  Costs 

Despite  the  potential  benefits  of  managing  O&S  cost  drivers,  the  research  team  has  found 
no  evidence  that  PMs  have  been  incentivized  in  this  area  in  the  past.  The  acquisition  process 
tends  toward  optimism.  Requirements  are  frequently  established  without  a  full  understanding  of 
the  technical  and  cost  implications.  Contractors  are  confident  about  their  technical  capabilities 
and  tend  to  be  optimistic  about  what  they  can  accomplish.  Acquisition  PMs  are  success  oriented, 
reinforced  by  the  contractor’s  optimism.  These  tendencies  translate  into  programs  faced  with 
areas  of  high  risk  because  integrating  advanced  technology  into  operational  capabilities  is  often 
harder  than  anticipated. 

When  such  risks  materialize,  the  PM  is  motivated  to  solve  the  problem.  Requesting  that  a 
requirement  be  relaxed  is  likely  going  to  be  the  last  resort,  since  it  may  be  considered  a  failure. 
Most  likely,  the  problem  must  be  addressed  by  the  contractor  design  teams  and  engineers.  In 
fact,  the  contractor  will  normally  learn  about  a  technical  problem  before  the  program  office.  The 
contractor  makes  a  profit  by  selling  products,  and  the  best  way  to  do  so  is  to  deliver  what  the 
customer  wants.  Therefore,  if  design  trades  are  needed  to  solve  the  problem,  the  contractor  will 
make  the  trades  based  on  its  perception  of  customer  priorities.  The  relatively  short  tenure  of 
acquisition  PMs  means  they  will  probably  not  be  involved  when  high  O&S  costs  are  incurred. 
Unless  the  government  makes  O&S  costs  an  explicit  priority,  it  is  relatively  easy  for  industry  to 
largely  ignore  O&S  costs  when  making  tradeoffs  associated  with  inevitable  near-term  technical 
issues.  In  most  cases,  performance,  acquisition  cost,  and  schedule  are  the  highest  PM  priorities, 
and  the  case  for  “fixing  O&S  later”  is  easily  rationalized.  The  government  processes,  as 
described  above,  if  implemented  effectively,  should  serve  to  counter  these  tendencies. 

Reducing  O&S  costs  early  in  design  will  likely  involve  extra  cost  and  time.  For  example, 
improving  system  reliability  may  require  improving  the  reliability  of  multiple  components  in  the 
system.  Lowering  O&S  costs  early  may  also  decrease  performance,  for  example,  if  it  is 
necessary  to  add  weight.  During  sub-component  selection,  the  contractor  will  normally  make 
decisions  based  on  its  perception  of  the  program’s  priorities.  Consider  a  situation  where  the 
contractor  has  a  choice  between  two  parts  where  one  part  likely  will  just  meet  reliability 
requirements  and  another  is  much  better  but  more  expensive.  Industry  can  be  expected  to  choose 
the  cheaper  part  unless  there  is  an  incentive  to  do  otherwise.  Because  of  enormous  pressure  to 
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reduce  acquisition  cost,  few  programs  would  entertain  a  trade  that  increases  investment  costs  to 
achieve  future  O&S  cost  savings.  Such  tradeoffs  are  likely  to  involve  performance  requirements 
or  operational  parameters,  so  involvement  of  the  operational  community  is  essential.  An 
effective  working  relationship  among  the  operational  community,  the  PM,  and  the  contractor  is 
necessary. 

Successful  implementation  of  affordability  goals  and  caps  involves  changing  this  behavior 
by  assigning  a  high  priority  to  managing  controllable  O&S  cost  drivers.  There  are  two  major 
elements  in  changing  such  systemic  behavior.  Changing  the  government’s  behavior  will  change 
industry  behavior.  Industry  behavior  will  be  most  affected  by  RFPs,  source  selection  criteria, 
contract  requirements,  and  contract  incentives  along  with  reinforcing  behavior  by  the 
government. 

In  weapon  system  acquisition,  there  is  often  competition  through  TMRR  and  a  down-select 
made  for  Milestone  B.  As  discussed  above,  the  period  between  Milestone  A  and  Milestone  B  has 
an  enormous  impact  on  both  investment  and  O&S  costs.  In  formulating  a  proposal  in  response  to 
a  pre-Milestone  B  RFP,  industry  will  consider  various  conceptual  design  alternatives.  The 
alternative  believed  to  have  the  best  chance  of  winning  the  EMD  contract  will  be  the  one 
selected  for  further  development.  The  government  can  reduce  O&S  costs  by  incentivizing 
industry  to  reduce  O&S  cost  during  conceptual  design.  Once  the  design  is  in  place,  resources 
should  be  allocated  to  making  sure  that  estimated  values  of  the  O&S  cost  drivers  are  in  fact 
achieved  and  to  seeking  ways  to  reduce  them  further  where  feasible. 

The  following  are  suggestions  for  providing  such  motivations: 

•  Explicitly  state  in  RFPs  that  the  program  is  driven  by  life-cycle  costs. 

•  Heavily  weight  O&S  cost  affordability  in  source  selection  criterion. 

•  Perform  early  systems  engineering  (by  the  government)  to  inform  evaluation  of 
proposals. 

•  Provide  a  wide  enough  range  between  threshold  and  preferred  values  for  performance 
criteria  to  enable  real  trades  with  O&S  cost  during  conceptual  design. 

•  Include  the  O&S  cost  affordability  goal  as  the  upper  limit  for  an  O&S  cost  threshold. 

•  Require  the  contractor  to  show  conceptual  design  tradeoffs  to  provide  the  government 
with  the  opportunity  to  widen  the  trade  space  if  it  finds  that  O&S  cost  reduction 
opportunities  are  not  being  developed. 

•  Identify,  based  on  experience  with  legacy  systems,  the  controllable  O&S  cost  drivers, 
thereby  defining  the  areas  where  conceptual  design  O&S  cost  reduction  efforts  should 
concentrate. 

•  Require  TMRR  and  EMD  contractors  to  construct,  maintain,  and  report  estimates  from 
an  accredited  life-cycle  cost  model.  (If  the  standard  O&S  cost  model  suggested  above  is 
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implemented,  the  eontractor’s  model  should  be  consistent  with  it,  though  probably  in 
greater  detail.) 

•  Require  contractors  to  identify  the  O&S  cost  drivers  in  conceptual  designs  and  to 
demonstrate  how  the  effects  of  those  cost  drivers  have  been  mitigated  at  the  preliminary 
design  review.  (The  TMRR  contractors  should  also  identify  the  controllable  O&S  cost 
drivers  for  which  cost  reduction  efforts  could  be  implemented  during  detailed  design.) 

•  Use  the  life-cycle  cost  model  to  effect  continuous  reductions  in  O&S  costs  throughout 
the  development  process. 

It  could  be  made  standard  practice  to  use  contract  incentives  to  reduce  O&S  cost.  Incentive 
fees  and  award  fees  are  two  examples  of  increasing  a  contractor’s  fee  as  a  function  of  specified 
criteria.  An  incentive  fee  is  determined  via  a  quantitative  metric.  For  example,  an  incentive  fee 
could  be  based  on  achieving  a  certain  level  of  reliability  to  be  demonstrated  in  particular  tests. 
Award  fees,  on  the  other  hand,  are  made  based  on  a  qualitative  determination  by  the  contracting 
officer  derived  from  opinions  of  subject  matter  experts  based  on  evidence  submitted  by  the 
contractor.  Award  fees  can  be  developed  for  improving  reliability  and  maintainability  in  general 
or  associated  with  some  particular  elements  of  the  system.  For  example,  the  contractor  can 
demonstrate  maintainability  by  designing  for  and  demonstrating  improvement  in  component 
removal  times.  In  addition,  award  fees  could  be  associated  with  the  contractor  trade  studies 
discussed  earlier  to  ensure  that  O&S  costs  are  considered.  While  such  techniques  have  been 
employed  in  some  DOD  acquisition  programs,  these  tools  have  not  been  widely  used. 

A  Value  Engineering  Change  Proposal  (VECP)  is  a  proposal  submitted  by  a  contractor 
under  the  Value  Engineering  provisions  of  the  Eederal  Acquisition  Regulation  that,  through  a 
change  in  the  contract,  would  lower  the  program’s  life-cycle  cost  to  DOD.  Contractors  are 
incentivized  to  do  this  because  they  can  share  in  the  savings  (up  to  75  percent).  VECPs  should  be 
encouraged,  and  consideration  of  O&S  cost  savings  should  be  given  equal  weight  to  investment 
costs.  Currently  VECPs  have  been  for  relatively  modest  monetary  amounts;  however,  wider 
usage  could  potentially  generate  much  greater  savings. 

Such  incentives  could  lead  to  the  selection  of  higher  reliability  parts  and/or  designs  that 
improve  maintainability.  The  incentives  should  apply  throughout  the  entire  EMD  period  even 
though  most  design  decisions  are  made  prior  to  the  Critical  Design  Review.  Between  Critical 
Design  Review  and  Eow-rate  Initial  Production  (TRIP),  there  is  intense  pressure  on  schedule, 
often  to  the  detriment  of  pursuing  additional  O&S  cost  reduction  activities.  The  proper  incentive 
structure  will  encourage  the  contractor  to  focus  on  both. 


One  example  is  the  F-16  program. 
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£.  Institutionalization 

An  ongoing  commitment  from  senior  leadership  is  neeessary  for  any  new  initiative  to  have 
a  long-term  impaet.  This  is  espeeially  important  for  initiatives  involving  O&S  eosts  for  new 
aequisition  programs  beeause,  as  diseussed  earlier,  it  is  likely  that  aequisition  managers  will  give 
lower  priority  to  O&S  eoneems  when  making  tradeoffs  assoeiated  with  inevitable  near-term 
teehnieal  issues.  A  eommitment  from  senior  leaders  is  more  than  words,  poliey,  and  guidanee. 
Effeetive  proeesses  on  paper  will  not  ehange  deeply  ingrained  behaviors.  Senior  leaders  must  be 
willing  to  foree  the  tough  deeisions  and  make  the  hard  ehoiees  assoeiated  with  aehieving  the 
desired  ends. 

While  strong  eommitment  of  senior  leadership  to  affordability  goals  and  eaps  appears  to 
exist  today,  the  lesson  of  history  is  that  it  is  diffieult  to  aehieve  long-term  institutionalization. 
Changes  in  leadership  almost  always  lead  to  new  areas  of  emphasis.  For  example,  CAIV 
originated  in  the  1994-1997  time  frame.  For  the  next  four  years,  there  was  very  little  publie 
mention  of  CAIV  by  senior  leadership  although  former  DOD  personnel  have  told  the  researeh 
team  that  programs  were  privately  eneouraged  to  eontinue  pursuit  of  the  initiative  in  not-for- 
attribution  reviews  of  CAIV  plans.  Similarly,  DTC,  though  deeply  entrenehed  in  aequisition 
poliey,  apparently  failed  to  reeeive  eontinued  emphasis  by  aequisition  managers. 

The  eommitment  of  senior  leadership  must  be  in  plaee  long  enough  for  the  permanent 
workforee  implementing  the  initiative  to  realize  the  benefits.  That  will  minimize  the  likelihood 
that  the  next  leadership  team  will  undo  the  ehanges.  Instantiation  of  poliey,  guidanee,  and 
oversight  is  a  neeessary  starting  point.  The  evidenee  that  this  is  oeeurring  for  the  BBP  initiatives 
is  as  follows: 

•  The  requirement  to  establish  and  enforee  affordability  goals  and  eaps  was  ineluded  in 
the  reeently-published  revision  of  DODI  5000.02. 

•  An  affordability  goals  and  eaps  seetion  has  been  ineluded  in  the  Defense  Acquisition 
Guidebook. 

•  A  revised  Operating  and  Support  Cost-Estimating  Guide  has  been  published. 

•  An  Operating  and  Support  Cost  Management  Guide  is  under  review. 

•  The  OUSD(AT&F)  offiee  of  Fogisties  &  and  Materiel  Readiness  (F&MR)  has 
established  systematie  interfaees  with: 

-  CAPE  on  the  independent  eost  estimates  required  at  Milestone  A  and  Milestone  B; 

-  Systems  Engineering  as  part  of  the  Program  Support  reviews;  and 

-  The  DAES  proeess  to  highlight  O&S  eost  issues. 

While  it  is  relatively  easy  to  establish  poliey  and  guidanee,  this  initiative  is  still  far  from  the 
degree  of  institutionalization  required  to  ehange  behavior.  As  evidenee,  RFPs  are  still  being 
issued  without  adequate  emphasis  on  O&S  affordability.  The  researeh  team  has  reviewed  RFPs 
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for  the  Army’s  Ground  Combat  Vehicle  (GCV)  and  Armored  Multipurpose  Vehicle  (AMPV)  for 
focus  on  control  of  O&S  costs. 

The  GCV  TMRR  RFP  affordability  sections  stated:  “Any  proposal  received  in  response  to 
this  RFP  that  proposes  a  price  in  excess  of  $450,000,000  [for  the  technology  development 
contract]  will  be  considered  unaffordable.”  There  is  nothing  about  O&S  affordability.  The 
program  intent  section  does  refer  to  both  acquisition  and  O&S  cost  affordability  but  as  a 
secondary  consideration.  The  RFP  stated: 

Offerors  must  balance  Affordability  with  the  achievement  of  the  “Big  Four” 
imperatives.  The  Government's  average  Unit  Manufacturing  Cost  target  for  the 
GCV  IFV  Program  is  $9,000,000  to  $10,500,000  per  unit  expressed  in 
Government  fiscal  year  2010  constant  dollars  (as  defined  by  the  2010  Office  of 
the  Secretary  of  Defense  Inflation  Guidance  FY2011  President’s  Budget).  In 
addition,  the  GCV  IFV  target  Operation  &  Sustainment  cost  is  $200  per  mile 
expressed  in  Government  fiscal  year  2010  constant  dollars  (as  defined  by  the 
2010  Office  of  the  Secretary  of  Defense  Inflation  Guidance  FY2011  President’s 
Budget). 

The  research  team  reviewed  several  charts  provided  by  the  Army  for  DAB  reviews  that 
displayed  the  O&S  costs  of  future  ground  combat  systems.  The  charts  indicate  that  the  Army’s 
program  for  the  systems  being  displayed  (including  GVC  and  AMPV)  is  not  affordable  for  both 
investment  and  O&S  costs.  GCV  is  the  principal  driver  of  those  O&S  cost  projections. 

There  is  language  in  the  EMD  AMPV  RFP  about  O&S  affordability:  “The  Government  has 
determined  the  Cost  Per  Mile  (consumables,  reparables,  and  fuel  cost  per  mile)  threshold  target 
for  the  AMPV  Family  of  Vehicles  (FoV)  is  $90  per  operating  mile  (Fiscal  Year  13  constant 
dollars).”  This  is  not  a  very  comprehensive  treatment  of  the  subject.  However,  the  AMPV’s 
O&S  costs  are  not  especially  high  compared  to  the  GCV.  Since  there  is  no  evidence  that  the 
O&S  costs  of  the  rest  of  the  portfolio  is  under  control,  a  decision  to  proceed  gives  the  appearance 
of  having  ignored  O&S  affordability. 

F.  Summary 

The  following  points  summarize  the  insights  of  this  chapter: 

•  Monitoring  affordability  goals  and  caps  is  necessary  but  not  sufficient  to  guarantee 
success. 

•  Improving  O&S  cost  tradeoff  processes  would  be  beneficial. 


U.S.  Army,  Solicitation  Number  W56HZV-1 1-R-OOOl  “GCV  TMRR  RFP”  (Warren,  MI:  U.S.  Army 
Contracting  Command,  30  November  2010),  3. 

Ibid.,  4. 

U.S.  Army,  Solicitation  Number  W56HZV-13-R-0022  “EMD  AMPV  RFP”  (Warren,  MI:  U.S.  Army 
Contracting  Command,  n.d.),  37. 
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•  There  must  be  parallel  efforts  to  manage  O&S  eost  drivers  from  both  industry  and 
Government  perspeetives. 

•  There  should  be  eontraet  ineentives  to  eneourage  sueh  aetivities. 

•  Irrespeetive  of  the  virtues  of  the  framework  put  in  plaee,  history  has  amply 
demonstrated  that  OSD  management  must  assiduously  and  relentless  enforee 
eomplianee  with  direetives  for  real,  long-term  ehange  to  oeeur.  Programs  of  dubious 
affordability,  either  investment  or  O&S,  should  not  be  approved  until  sueh  issues  are 
resolved. 
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Appendix  A 

Case  Studies  of  the  Application  of  the  Cost  as  an 
Independent  Variable  (CAIV)  Concept 


This  appendix  is  supplementary  to  the  discussion  of  Cost  as  an  Independent  Variable 
(CAIV)  programs  in  Chapter  2,  Section  C.  For  the  twelve  programs  using  CAIV  that  were 
examined  in  this  research,  the  appendix  will  present 

•  the  cost  history  of  each  program; 

•  the  reasons,  as  best  understood,  why  that  history  occurred;  and 

•  an  assessment  of  whether  CAIV  principles  were  actually  applied  in  the  program,  and  if 
so,  with  what  success. 

To  speed  understanding  and  application  of  acquisition  reform,  the  December  4,  1995 
Kaminski  memorandum^  designated  eight  Major  Defense  Acquisition  Programs  (MDAP)  as 
Flagship  Pilot  Programs  (FPP)  for  CAIV  and  certain  other  initiatives.^  The  eight  FPPs  are 

•  U.S.  Army — Army  Tactical  Missile/Brilliant  Anti-Tank  (ATACMS/BAT),  Crusader 
self-propelled  artillery  systems; 

•  U.S.  Navy  (USN) — Multifunctional  Information  Distribution  System  (MIDS), 
Sidewinder  air-to-air  missile  (AIM-9X);  and 

•  U.S.  Air  Force  (USAF) — Space-Based  Infrared  System  (SBIRS),  Joint  Air-to-Surface 
Standoff  Missile  (JASSM),  Joint  Strike  Fighter  (JSF/F-35),  and  an  Evolved  Expendable 
Eaunch  Vehicle  (EEEV). 

Each  of  these  EPPs,  as  well  as  four  other  prominent  programs  that  advertised  use  of  CAIV, 
will  be  reviewed  in  this  appendix  to  help  to  fill  out  the  picture  of  the  application  of  CAIV 
principles.  The  four  additional  programs  are 

•  USAE — Joint  Direct  Attack  Munition  (JDAM) 

•  USN — Eittoral  Combat  Ship  (ECS),  Amphibious  Transport  Dock  (EPD-17) 

•  U.S.  Marine  Corps  (USMC) — Expeditionary  Eighting  Vehicle  (EEV) 


Paul  Kaminski,  Memorandum  for  Distribution  Subject:  Reducing  Life-Cycle  Costs  for  New  and  Fielded 
Systems,  December  4,  1995.  No  copy  of  this  memo  has  been  found  but  the  text  of  it  is  at 
http://www.navair.navv.mil/nawctsd/Resources/Librarv/Acqguide/caainvar.htm. 

How  those  programs  were  chosen  was  not  specified,  but  it  is  likely  that  they  were  nominated  by  the  Services. 
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The  Selected  Acquisition  Reports  (SAR)  have  been  used  for  the  principal  cost  measures 
that  CAIV  addresses,  program  acquisition  unit  cost  (PAUC)  and  average  procurement  unit  cost 
(APUC),  and  for  information  on  cost  thresholds  in  the  acquisition  program  baseline  (APB)  for 
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each  program.  The  SAR  analyses  of  cost  variances  might  be  considered  a  valuable  source  for 
this  analysis,  but  unfortunately  that  information  has  been  shown  to  be  of  very  limited  value. 

Fortunately  several  of  the  programs  of  interest  have  been  addressed  in  considerable  depth  in 
a  previous  IDA  analysis:  Gene  H.  Porter  et  ah.  The  Major  Causes  of  Cost  Growth  in  Defense 
Acquisition,  IDA  Paper  P-4531,  December  2009,  which  will  be  referred  to  subsequently  as  “P- 
4531.”  The  current  research  will  draw  from  that  analysis  extensively.  To  get  the  full  story,  the 
reader  is  encouraged  to  review  relevant  material  in  Volume  III  of  P-4531.  For  programs  not 
treated  in  P-4531,  this  examination  relies  on  readily  available  documentary  sources,  primarily 
SARs. 

JDAM  and  JASSM  are  addressed  first  because  of  the  pivotal  roles  they  played  in  CAIV  and 
because  the  two  programs  were  closely  linked.  ^ 

A.  JDAM 

JDAM  was  not  a  CAIV  FPP,  but  it  began  as  a  Defense  Acquisition  Pilot  Project  (DAPP) 
with  a  very  strong  cost  control  focus  and  was  influential  in  thinking  about  CAIV.  It  became  a 
CAIV  program  as  CAIV  took  shape,  with  strong  backing  from  high  level  officials  encouraging 
requirements  tradeoffs  to  meet  stringent  cost  goals.  ^ 

The  program  designed  and  produced  an  all-weather  glide-bomb  kit  that  could  be  fitted  to 
commodity  bomb  bodies  to  make  them  into  moderate-precision,  relatively  inexpensive,  free-fall 
weapons.  At  the  time  it  was  initiated  in  the  early  1990s,  the  United  States  had  five  decades  of 
experience  in  developing  and  employing  guided  bombs,  from  the  AZON  [azimuth  only]  and 
RAZON  [range  and  azimuth]  kits  of  World  War  II  through  the  laser-guided  bomb  kits  that  began 
in  the  1960s.  It  was  a  class  of  weapons  well  understood  by  users  and  weapons  engineers,  leading 
to  implicit  expectations  about  engineering  approaches  and  costs.  It  seemed  likely  that  pursuing 
the  program  along  established  lines  would  lead  to  a  unit  procurement  cost  in  excess  of  $40,000. 


For  definitions  of  precisely  what  is  and  is  not  included  in  PAUC  and  APUC,  see 
https://dap. dau.mil/aap/pages/qdetails. aspx?cgiSubiectAreaID=15&cgiQuestionID=l  6702. 

Paul  G.  Hough,  Pitfalls  in  Calculating  Cost  Growth  from  Selected  Acquisition  Reports,  N-3 136-AF  (Santa 
Monica,  CA:  RAND  Corp.,  1992).  More  recent  informal  analyses  conducted  by  OUSD(AT&L)  reveal  that  this 
situation  has  not  measurably  improved. 

Col.  Terry  R.  Little,  the  program  manager  for  JDAM  in  the  mid-1990s,  subsequently  became  the  program 
manager  for  JASSM  in  the  late-1990s. 

For  an  extended  history  of  the  initial  years  of  JDAM,  see  Chynthia  Ingols  and  Lisa  Brem,  Implementing 
Acquisition  Reform:  A  Case  Study  on  Joint  Direct  Attack  Munitions  (Fort  Belvoir,  VA:  Defense  Systems 
Management  College,  July  1998),  http://acquisition.gov/sevensteps/librarv/JDAMsuccess.pdf  Additional 
information  and  perspectives  are  provided  in  Mark  A.  Lorell  and  John  C.  Graser,  An  Overview  of  Acquisition 
Reform  Cost  Savings  Estimates,  MR-1329  (Santa  Monica,  CA:  RAND  Corp.,  2001). 
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However,  analogs  to  key  guidanee  system  eomponents  were  available  in  eommereial  markets 
where  priees  were  deelining  rapidly.  For  nearly  a  deeade,  the  USN  had  exploited  similar 
situations  to  aehieve  substantial  eeonomies  in  sonobuoy  priees,  and  there  was  optimism  in  some 
quarters  about  gaining  eomparable  results  in  JDAM,  despite  its  greater  complexities. 

In  mid-1995  the  JDAM  Joint  Operational  Requirements  Document  (JORD)  was  revised  to 
incorporate  a  unit  production  price  of  no  more  than  $40,000  base  year  1995  dollars  (BY  1995  $) 
as  a  requirement,  with  an  objective  of  $30,000  or  less.  $40,000  was  moderately  lower  than 
contemporary  independent  cost  estimates,  and  $30,000  was  substantially  lower. ^ 

As  a  DAPP,  JDAM  won  permission  to  use  not  only  commercial  technologies  but  to  enter 
into  commercial  kinds  of  relationships  with  contractors,  involving  greatly  reduced  government 
oversight  and  control  and  relying  on  performance-oriented  specifications  and  strong  financial 
incentives.  At  that  time  there  were  still  a  number  of  defense  firms  that  were  able  to  compete,  and 
the  program  office  made  effective  use  of  competitive  pressures.  It  is  unclear  how  well  that 
approach  would  work  in  today’s  environment  of  greatly  narrowed  competition. 

Ultimately  both  of  the  two  final  competitors  offered  JDAM  designs  that  used  commercial 
and  commercial-grade  parts,  and  priced  them  on  a  commercial  basis  (i.e.,  well  down  the  learning 
curve  and  taking  a  loss  on  initial  lots  while  turning  profits  on  later  lots). 

For  the  competing  contractors,  JDAM  offered  an  attractive  risk/reward  calculus.  A 
contractor  that  underestimated  production  costs  faced  a  risk  of  significant  loss  on  the  early 
production  lots,  but  was  under  little  obligation  to  continue  to  produce  at  a  loss  in  the  long  run, 
and  if  the  contractor  declined  to  re-bid  at  a  low  price  for  subsequent  lots,  the  government  might 

o 

find  itself  in  a  difficult  situation.  On  the  other  hand,  if  the  contractor’s  winning  bid  price  proved 
to  be  realistic,  he  stood  to  make  super-normal  profits  by  selling  large  numbers  of  JDAM  kits  on 
commercial  terms  over  many  years — as  proved  to  be  the  case.  Thus  there  was  downside  risk,  but 
the  upside  potential  was  much  greater. 

The  program  has  been  highly  successful  in  both  cost  and  performance.  The  original 
(development)  APB  APUC  was  $31,000  in  fiscal  year  1995  dollars  (FY1995  $)  based  on 
procurement  of  87,496  units.  The  December  2012  SAR  reports  a  total  production  of  208,613 
units  to  date  with  an  APUC  of  $19,800,  again  in  FY1995  dollars.  Thus  the  APUC  has  decreased 
by  28  percent,  or  21  percent  when  adjusted  for  the  change  in  quantity.  By  any  measure,  JDAM 
has  been  a  signal  success  for  the  CAIV  concept. 


Lorell  and  Graser,  An  Overview  of  Acquisition  Reform  Cost  Savings  Estimates,  MR-1329,  66-61 . 
Ibid.,  62-63. 
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B.  JASSM  (Baseline  variant  only) 

The  JASSM  program  was  initiated  in  1995  after  the  highly  elassified  Tri-Serviee  Standoff 
Attaek  Missile  (TSSAM)  program  was  caneelled,  largely  beeause  of  a  substantial  cost  overrun.^ 
Given  that  history  and  the  contemporary  success  in  cost  control  of  the  JDAM  program,  there  was 
great  optimism  that  taking  the  same  approach  on  JASSM  would  lead  to  similar  results.  In  2009, 
the  program  was  divided  into  two  subprograms — the  baseline  missile  and  an  extended  range 
(ER)  variant.  This  paper  will  only  discuss  the  baseline  missile  in  any  detail.  The  baseline  missile 
is  a  subsonic  cruise  missile  with  200nmi  range,  GPS/inertial  guidance  plus  infrared  (IR)  terminal 
homing,  a  1,000  lb.  warhead,  and  low  observables.  It  is  thus  substantially  more  complex  than 
JDAM,  but  was  intended  to  utilize  much  the  same  acquisition  strategy,  with  the  same  initial 
program  manager  who  had  set  JDAM  on  its  successful  course.  It  was  a  CAIV  flagship  program 
and  CAIV  was  an  essential  part  of  its  acquisition  strategy. 

Unlike  JDAM,  competition  was  not  maintained  in  the  program.  An  early  design 
competition  between  McDonald  Douglas  and  Lockheed  Martin  was  won  by  the  latter,  which  was 
selected  as  the  prime  contractor  for  the  Program  Definition  and  Risk  Reduction  (PDRR)  phase 
and  in  a  subsequent  competition  for  the  full-scale  development  contract.  The  original  plan  was  to 
continue  competition  at  least  through  PDRR;  however,  funding  constraints  curtailed  those  plans. 
Prior  to  its  entry  into  engineering  development  at  acquisition  Milestone  11  in  mid-1998,  the 
program  initially  seemed  to  enjoy  substantial  success  in  controlling  cost  by  allowing  and 
incentivizing  the  contractor  to  make  broad  requirements  tradeoffs.  It  was  a  period  characterized 
by  very  high  enthusiasm  for  acquisition  reform,  and  the  program  was  structured  to  employ  many, 
if  not  virtually  all,  of  those  initiatives  including 

•  streamlining  the  acquisition  process — lower  frequency  of  management  reviews,  reduced 
documentation,  and  less  submission  of  data  that  facilitated  oversight; 

•  using  more  commercial  practices; 

•  giving  the  contractor  more  flexibility  in  product  design,  development  and  production; 
and 

•  implementing  CAIV. 

The  promise  of  a  missile  with  substantially  lower  costs  though  was  short-lived,  and  soon 
after  the  Milestone  11  decision,  costs  began  to  grow,  as  seen  in  Figure  A-1  extracted  from  P- 
4531.^° 


JASSM  is  one  of  the  programs  examined  in  considerable  detail  in  Vol.  Ill,  Appendix  F,  of  Gene  H.  Porter  et  al., 
The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  IDA  Paper  P-453 1  (Alexandria,  VA:  Institute  for 
Defense  Analyses  (IDA),  December  2009).  This  section  is  largely  drawn  from  that  paper,  which  the  reader  is 
encouraged  to  consult  for  a  much  more  detailed  discussion. 

Porter  et  al..  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  Vol.  Ill,  F-I5. 
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APUC 

(FY98$K) 

Year-to- 

Year 

Change 

Aggregate 

Change 

Quantity 

Dec  99 

399 

2400 

Dec  01 

448 

12.3% 

12.3% 

3700 

Dec  02* 

523 

16.7% 

31.1% 

4340 

Dec  03 

525 

0.4% 

31.6% 

4366 

Dec  04 

541 

3.0% 

35.6% 

4900 

Dec  05 

563 

4.1% 

41.1% 

4900 

Dec  06 

649 

15.3% 

62.7% 

4900 

Dec  07 

703 

8.3% 

76.2% 

4900 

*  The  Dec-02  (and  subsequent  SARs)  reflect  ER  requirement 
(quantity  not  specified) _ 


Figure  A-1.  JASSM  APUC  Cost  Growth 


As  explained  in  P-4531,  Figure  A-1  reflects  both  growth  in  the  cost  of  the  baseline  missile 
and  the  addition  of  the  ER  variant  in  2002.  In  2009  the  program  declared  a  Nunn-McCurdy 
breach.  The  subsequent  analysis  by  the  Cost  Analysis  Improvement  Group  (CAIG)  showed  that 
by  that  time  the  APUC  for  just  the  baseline  version  had  grown  to  $688  thousand  (the  estimate  in 
the  Dec  2012  SAR  is  $935  thousand  in  FY2010,  corresponding  to  about  $722  thousand  in 
FY1995  $).  This  is  remarkably  close  to  the  original  cost  objective  for  TSSAM,  whose  unit  cost 
had  grown  to  over  two  million  when  it  was  cancelled. 

Except  for  maintaining  competition,  the  JASSM  program  was  structured  much  like  JDAM. 
However,  the  other  acquisition  reform  steps  that  had  worked  so  well  for  JDAM  were 
unsuccessful  when  applied  to  the  JASSM  program,  in  fact  dramatically  so.  CAIV-directed 
efforts,  in  particular,  resulted  in  significant  problems.  For  example  the  use  of  commercial 
components  to  drive  down  costs,  some  from  small  companies  with  little  or  no  experience  in 
producing  weapon  systems,  contributed  to  reliability  issues. 

Thus  the  CAIV  strategy  for  JASSM  was  remarkably  unsuccessful,  not  only  in  failing  to 
control  costs  but  in  producing  a  missile  with  significant  effectiveness  issues  as  well,  reliability  in 
particular.  The  quote  below  is  taken  from  P-4531  (page  F-1 1): 

Factors  contributing  to  JASSM ’s  poor  reliability  can  be  traced  back  to  the  way  the 
program  was  formulated  and  executed,  as  detailed  above.  The  aggressive  use  of 
CAIV  led  to  significant  cost  cutting,  consequently  resulting  in  an  insufficiently 
robust  missile  design;  poor  systems  engineering;  weak  manufacturing  controls, 
badly  managed  supply  chains;  inadequate  live  fire  testing,  and  over  reliance  on 
modeling  and  simulation.  As  already  noted,  TSPR  [Total  Systems  Performance 
Responsibility]  gave  the  EM  [Lockheed  Martin]  virtually  total  control  over 
decisions  in  making  trades  between  costs  and  missile  performance,  as  well  as 
exclusive  authority  over  the  development  and  execution  of  developmental  testing. 
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That  resulted  in  the  heavy  reliance  on  modeling  and  simulation  to  predict  missile 
mission  effectiveness  and  reliability,  while  cutting  back  on  costly  but  much  more 
meaningful  live  fire  missile  testing. 

Fortunately  it  appears  that  the  reliability  problems  are  at  last  under  control,  and  the  APUC 
estimate  has  actually  decreased  modestly  in  more  recent  SARs.  While  one  might  say  “all’s  well 
that  ends  well,”  there  was  a  great  deal  of  costly  program  turbulence  which  might  have  been 
avoided  and,  more  importantly,  delays  in  fielding  the  weapon  that  could  have  had  serious 
operational  implications  had  U.S.  forces  become  involved  in  a  contingency  that  required 
JASSM’s  capabilities. 

A  synthesis  of  the  major  causes  of  acquisition  cost  growth  in  the  JASSM  program  described 
in  IDA  Paper  P-4531  is  summarized  in  Figure  A-2.^^ 


•  Over-ambitious  acquisition  reform  objectives  — ”50%  savings”  (cost/scheduie) 

-  Placed  undue  pressure  to  achieve  unrealistic  objectives  (cost,  schedule  and  management  reforms) 

-  Vested  too  much  control  with  contractor  with  potential  conflict  of  interests  (TSPR) 

-  FFP  ill-suited  for  ambitious  development  with  unproven  acquisition  strategy 

-  Insufficient  PM  visibility  and  too  little  OSD  oversight) 

-  Inadequate  development  flight  testing  and  misapplication  of  modeling  &  simulation 

•  CAIV’s  early  cost  goals  contributed  to  poor  contractor  performance  (poor  supplier 
management,  manufacturing  quality,  systems  and  reliability  engineering) 

•  Lack  of  consistent  missile  reliability  forced  change  in  requirements  specifications 

-  Original  requirement  based  on  simulation-based  assessments  of  mission  effectiveness  (55  missiles  to 
kill  17  targets )('wb;cb  was  achieved!) — no  explicit  reliability  objective  specified 

-  When  lot  testing  revealed  reliability  issues,  government  imposed  a  reliability  criterion  that  the  contractor 
has  not  yet  been  able  to  meet  consistently 

-  Resulted  in  lower  funding  and  production  rate  and  thus  increased  unit  costs 

•  Unrealistic  unit  cost  estimates  at  MS  -II  (62%  APUC  cost  growth  at  breach) 

-  At  MS-II  CAIG  APUC  estimate  of  baseline  missile14%  above  Service  estimate 

-  At  Nunn-McCurdy  breach,  CAIG  APUC  estimate  was  50%  higher  than  at  MS-  II 

•  Post  MS-II  introduction  of  requirement  for  a  more  costly  extended  range  missile 

-  Could  have  been  mitigated  by  separate  tracking  of  ER  missile  cost 


Note:  PM  =  Program  Manager;  OSD  =  Office  of  the  Secretary  of  Defense;  MS  =  milestone. 

Figure  A-2.  Reasons  for  JASSM  Cost  Growth  from  IDA  Paper  P-4531 

While  JDAM  demonstrated  that  when  properly  and  appropriately  employed,  the  CAIV 
approach  could  be  a  powerful  cost-control  tool,  JASSM  by  contrast  illustrates  the  many  hard-to- 
foresee  pitfalls  that  the  overly-zealous  application  of  CAIV  could  encounter,  even  with 
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experienced  and  expert  direction.  Another  lesson  to  be  learned  from  JASSM  is  the  importance  of 
a  sound  initial  cost  estimate  that  can  serve  as  a  valid  basis  for  setting  a  CAfV  cost  goal.  It  is 
ironic  that  the  current  APUC  for  the  baseline  JASSM  is  remarkably  close  to  early  cost  estimates 
of  about  $700,000  for  a  missile  with  JASSM’s  characteristics,  before  the  ambitious  claims  of 
much  lower  costs  based  on  acquisition  reform  initiatives  (CAIV,  among  others)  led  to 
embedding  a  figure  over  40  percent  lower  in  the  APB.  It  is  also  an  excellent  illustration  of  the 
dangers  of  setting  overly  aggressive  cost  targets.  If  both  DOD  program  managers  and  contractors 
are  strongly  incentivized  to  undertake  “heroic”  efforts  to  reduce  costs,  their  risk-reward  calculus 
may  translate  into  excessive  risks  to  DOD  and  the  warfighter.  There  is  good  evidence  that 
JASSM  is  a  case  in  point. 

C.  SBIRS 

SBIRS  employs  a  constellation  of  satellites  in  geosynchronous  earth  orbit  (GEO)  and  high- 
elliptical  orbit  (HEO)  to  maintain  continuous  multiband  IR  surveillance  of  the  great  majority  of 
Earth’s  surface,  including  the  entire  Northern  Hemisphere,  with  the  primary  objective  of 
providing  very  fast  alerting  of  ballistic  missile  launches  and  initial  trajectory  predictions.  Other 
capabilities  include  observation  of  space  launches  and  shorter-ranged  ballistic  missiles,  as  well 
as  tactical  weapons.  It  supplements  and  eventually  is  to  replace  the  Defense  Support  Program 
(DSP)  constellation. 

The  SBIRS  case  was  analyzed  at  length  in  IDA  Paper  P-4531. Another  IDA  paper.  What 
to  Buy?  The  Role  of  Director  of  Defense  Research  and  Engineering  (DDR&E):  Lessons  from  the 
1970s,  which  reviewed  some  of  the  history  of  DSP,  SBIRS’  predecessor  program,  will  also  be 
used. 

DSP  (which  had  earlier  been  called  by  a  variety  of  code  names)  began  as  a  highly  risky  and 
extremely  urgent  effort  at  the  height  of  the  late-1950s  concerns  about  the  “missile  gap”  with  the 
Soviet  Union.  As  the  concerns  faded  (with  the  acquisition  of  better  intelligence),  development 
grew  more  deliberate  and  became  part  of  the  development  of  the  systems  engineering  approach, 
leading  to  design  simplification  and  substantially  lower  costs. 

The  key  factor  in  limiting  the  cost  was  use  of  a  one-dimensional  linear  sensor  array  that  was 
swept  across  the  field  of  regard  by  means  of  a  purely  passive  mechanism.  This  entailed  some 
performance  tradeoffs,  and  elements  in  the  USAE  sought  from  an  early  date  to  move  to  a  more 
sophisticated  and  costly  design  employing  two-dimensional  arrays.  Means  were  found  to  extend 


The  earliest  (“concept”)  APB  reflected  an  APUC  of  $710,000  in  FY1995  dollars.  The  “development”  APB 
reduced  it  to  $400,000. 

Porter  et  al..  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  Vol.  Ill,  Appendix.  J. 

William  D.  O’Neil  and  Gene  H.  Porter,  What  to  Buy?  The  Role  of  Director  of  Defense  Research  and 
Engineering  (DDR&E):  Lessons  from  the  1970s,  IDA  Paper  P-4675  (Alexandria,  VA:  IDA,  January  2011),  39- 
44. 
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the  performance  of  the  basic  DSP  system  while  further  improving  its  economy,  so  proposals  for 
a  follow-on  made  little  headway  until  the  1990s. 

Expectations  of  substantial  cost  reduction  through  employment  of  commercial  approaches 
and  hardware  elements  played  a  key  role  in  getting  SBIRS  started  in  the  1990s.  But  as  detailed  in 
P-4531  (Appendix  J),  these  expectations  faded  dramatically  after  2000. 

It’s  quite  difficult  to  present  a  clear,  simple  picture  of  SBIRS  costs,  as  the  structure  of  the 
acquisition  is  complex  and  varying,  and  the  mix  of  outputs  has  changed  frequently.  Figure  A-3 
shows  total  acquisition  program  costs  in  constant  dollars,  adjusted  for  quantity  changes.  As  of 
the  end  of  2012  the  total  acquisition  cost  had  increased  by  more  than  280  percent  since  program 
initiation  in  October  1996,  on  a  quantity-adjusted  basis. 


A-8 


3  m 
O*  3 

<  f 

re  >S 


_3 


1995 


2000 


2005 

Date  of  SAR 


2010 


2015 


Figure  A-3.  Estimates  of  Totai  SBiRS  Acquisition  Cost  for  Five  GEO  Sateiiites,  Two  HEO 
Packages,  and  Ground  Segment  (adjusted  as  necessary  for  quantity  changes) 


The  descriptions  of  the  early  thinking  about  SBIRS  seem  bizarre  in  retrospect.  Although  the 
environment  soon  gave  way  to  bitter  enmity  and  recriminations,  it  seems  evident  that  initially 
there  was  a  good  deal  of  real  harmony  and  unity  of  purpose  within  the  contractor  team  and 
between  the  contractor  and  government,  possibly  reflecting  that  competing  and  dissenting  voices 
were  being  shut  out.  The  systems  engineering  efforts  normally  relied  upon  heavily  in  space 
programs  to  provide  clarity  and  realism  were  deliberately  cut  back  very  severely  to  save  front- 
end  costs.  There’s  little  evidence  that  the  contractor  believed  the  initial  estimates  to  be  seriously 
unrealistic.  Nor  did  the  CAIG  challenge  the  cost  estimates. 

The  findings  in  IDA  Paper  P-453 1  regarding  cost  growth  are  summarized  in  Figure  A-4. 
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•  Competitive  pressures  motivated  both  government  and  industry  personnel  to 
promise  substantial  added  capability  at  a  cost  well  below  precedent 

•  Efforts  to  validate  and  refine  technical  plans  and  identify  risk  areas  through  early 
systems  engineering  were  inadequate  considering  the  complexity  of  the  concept 

•  Low  cost  estimates  based  on  projected  savings  from  "acquisition  reform" 
innovations  failed  to  materialize 

•  It  proved  technically  impossible  to  deliver  the  system  as  designed  within  the 
estimated  cost,  and  the  inadequate  funding  resulting  from  such  underestimates 
prompted  shortcuts  that  were  unsuccessful 

•  The  program  was  restructured  to  attack  the  problems  that  emerged,  but  the  costs 
of  remediation  were  themselves  underestimated 


Figure  A-4.  Findings  on  SBiRS  Cost  Growth 

While  there  appears  to  have  been  a  genuine  effort  to  utilize  the  CAIV  approach,  the 
emergence  of  serious  technical  issues  soon  after  Milestone  II  quickly  rendered  the  CAIV  cost 
objectives  irrelevant.  The  lack  of  sound  front-end  systems  engineering  had  a  heavy  impact  on  the 
reliability  of  the  initial  cost  estimates.  To  a  certain  extent  SBIRS  was  a  victim  of  the  same 
inflated  expectations  for  the  commercial  space  market  that  afflicted  EELV  cost  estimates.  It  also 
suffered  from  a  kind  of  stealth  requirements  creep,  as  officers  who  vastly  overestimated  the 
potential  for  cost  reduction  also  informally  pressed  for  capabilities  expansion  in  integrated 
product  teams  (IPT).  A  great  deal  of  growth  came  from  bad  engineering,  coupled  with 
abandonment  of  the  parallel  risk  reduction  development  policies  that  had  helped  keep  previous 
space  programs  on  track. 

In  any  event,  SBIRS  represents  another  dramatic  failure  of  CAIV.  As  with  JASSM,  there  is 
a  strong  lesson  that  CAIV  cannot  work  unless  the  cost  estimate  on  which  the  CAIV  cost 
objective  is  set  is  sound.  Good  early  systems  engineering  is  essential  to  establish  such  a  cost 
objective,  and  its  absence  seriously  hampered  SBIRS — much  more  so  that  JASSM,  a  far  simpler 
system.  SBIRS  is  thus  another  case  in  point  wherein  attempting  to  achieve  very  aggressive  cost 
reduction  goals  appears  to  have  contributed  to  diametrically  opposite  outcomes. 

D.  AIM-9X  Air-to-Air  Missile  BLOCK  I  AND  BLOCK  II 

The  AIM-9X  is  the  latest  variant  in  a  long  line  of  air-to-air  AIM-9  Sidewinder  short-range 
missiles  (SRM).  As  compared  with  the  preceding  AIM-9M,  AIM-9X  Block  I  provides 

•  an  expanded  kinematic  envelope  due  to  lower  airframe  drag  and  thrust-aided  steering; 

•  improved  sensor  performance  with  an  advanced  imaging  focal  plane  array  and  an 
advanced  processing  system; 

•  visual  cueing  via  a  helmet-mounted  sight  or  other  line-of-sight  source; 
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•  improved  countermeasures  resistance;  and 

•  increased  economy  of  operation  through  an  integral  onboard  cooling  system,  and 
various  detailed  improvements. 

It  also  provides  a  limited  lock-on-after-launch  (LOAL)  capability. 

AIM-9X  Block  II  goes  beyond  Block  I  in  incorporating  an  aircraft-to-missile  data  link  to 
provide  full  LOAL  capability  and  was  split  off  as  a  separate  MDAP  in  June  2011.  AIM-9X  was 
not  a  classic  acquisition  program  in  that  it  involved  incorporating  a  collection  of  improved 
subsystems  in  an  existing  system,  most  of  which  had  already  been  demonstrated  in  prototype 
form.  As  such  it  offered  good  prospects  for  a  successful,  if  not  particularly  challenging,  CAIV 
program,  since  major  risk-reduction  had  already  taken  place.  In  fact,  long  before  the  advent  of 
CAIV,  cost  had  been  a  focus  of  the  program  since  the  initial  requirement  was  approved  in  1990. 
CAIV  was  mentioned  in  the  initial  SAR  (December  1997)  but  not  thereafter. 

In  1996  Hughes  Electronics  won  the  Engineering  and  Manufacturing  Development  (EMD) 
contract  in  competition  with  Raytheon.  However,  as  part  of  the  ongoing  DOD  defense  contractor 
consolidation,  Raytheon  bought  Hughes  Electronics  (from  General  Motors)  in  1997.  Table  A-1 
presents  the  unit  cost  history  of  the  AIM-9X.  Development  generally  went  well,  with  fairly 
minor  problems  but  some  schedule  slippage.  External  factors  caused  further  schedule  slippage. 
Re-phasing  of  production  affected  estimated  costs.  There  were  two  Nunn-McCurdy  breaches  as  a 
result  of  these  factors,  but  neither  qualified  as  critical.  Performance  generally  met  goals. 
Reliability  substantially  exceeded  original  estimates,  benefitting  life  cycle  costs  (ECC). 

The  large  increase  in  PAUC  in  201 1  reflected  the  decision  to  split  the  Block  II  AIM-9X  off 
as  a  separate  MDAP.  This  meant  that  future  procurement  would  be  carried  under  the  Block  II 
rather  than  the  Block  I  program,  thus  decreasing  Block  I  quantity  and  increasing  its  program  unit 
cost.  APUC  increased  much  more  modestly,  reflecting  the  quantity  reduction.  (In  fact,  if  adjusted 
for  the  quantity  reduction,  APUC  actually  decreased.)  It  does  not  reflect  any  problem  with  Block 
1.  Eurther  production  will  be  of  the  Block  II,  which  is  more  expensive  due  to  the  incorporation  of 
the  data  link. 
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Table  A-1.  PAUC  and  APUC  for  AIM-9X  Block  I 


Costs  In  Budget  Year  1997  Dollars 


APB  Prodn. 
APUC  Qty. 


SAR 

Date 

PAUC 

Dec  1997 

258,000 

Dec  1998 

250,000 

Jun  1999 

250,000 

Dec  1999 

230,000 

Dec  2001 

239,000 

Dec  2002 

248,000 

Dec  2003 

249,000 

Jun  2004 

249,000 

Dec  2004 

242,000 

Dec  2005 

258,000 

Dec  2006 

261,000 

Dec  2007 

263,000 

Dec  2009 

282,000 

Dec  2010 

290,000 

Dec  2011 

421,000 

APB 

PAUC 

APUC 

245,000 

207,000 

245,000 

198,000 

245,000 

198,000 

245,000 

178,000 

245,000 

186,000 

245,000 

192,000 

245,000 

193,000 

245,000 

193,000 

245,000 

184,000 

252,000 

200,000 

252,000 

201,000 

252,000 

203,000 

252,000 

221,000 

282,000 

227,000 

282,000 

250,000 

193,000 

10,000 

193,000 

10,080 

193,000 

10,080 

193,000 

10,097 

193,000 

10,097 

193,000 

10,097 

193,000 

10,097 

193,000 

10,097 

193,000 

10,097 

193,000 

10,097 

193,000 

10,097 

193,000 

10,097 

193,000 

10,097 

221,000 

10,097 

221,000 

3,097 

The  AIM-9X  can  be  counted  as  a  cost-control  semi-success,  a  program  whose  costs  rose 
somewhat  (about  10  percent  overall),  but  by  less  than  the  average  for  MDAPs  of  this  class.  How 
much  that  outcome  can  actually  be  attributed  to  CAIV  policies  and  procedures  is  not  known; 
there  is  no  indication  of  the  extent  (if  any)  to  which  requirements  were  traded  off  to  hold  down 
costs. 

E.  MIDS 

MIDS  is  a  multinational  (United  States,  France,  Germany,  Italy,  Spain)  cooperative 
development  program  with  U.S.  Joint  Service  participation.  The  program  was  established  to 
design,  develop,  and  deliver  low  volume,  lightweight  tactical  information  system  terminals  for 
U.S.  and  Allied  fighter  aircraft,  bombers,  helicopters,  ships,  and  ground  sites.  The  MIDS 
program  consists  of  the  MIDS  Low  Volume  Terminal  (MIDS-LVT)  and  the  MIDS  Joint  Tactical 
Radio  System  (MIDS  JTRS)  terminal.  This  analysis  only  addresses  the  MIDS-LVT,  which  was  a 
CAIV  flagship  program.  MIDS  passed  Milestone  II  on  in  December  1993  but  was  not  designated 
a  CAIV  flagship  program  until  1997;  this  review  starts  at  that  point.  The  unit  cost  history  of 
MIDS  derived  from  the  SARs  is  displayed  in  Table  A-2. 


Despite  the  common  perception,  not  all  DOD  acquisition  programs  experience  cost  growth.  In  fact,  between 
their  initial  SARs  and  the  December  2009  SAR,  ten  out  of  ninety -one  MDAPs  showed  no  increase  in  APUC. 
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Table  A-2.  MIDS-LVT  Cost  and  Quantity  History,  as  Reported  in  SARs 

Costs  in  Base  Year  Dollars 


SAR  Date 

Base 

Year 

PAUC 

PAUC  in 
BY1992  $ 

APB 

PAUC 

Orig.  Baseline 
PAUC 

APUC 

APUC  in 
BY1992  $ 

APB 

APUC 

Orig. 

Baseline 

APUC 

Prodn 

Qty. 

Dec  1997 

1992 

500,000 

1,376,000 

261,000 

704,000 

2,358 

Dec  1998 

1992 

472,000 

500,000 

232,000 

261,000 

2,374 

Jun  1999 

1992 

472,000 

500,000 

232,000 

261,000 

2,374 

Dec  1999 

1992 

491,000 

500,000 

261,000 

261,000 

2,499 

Sep  2001 

1992 

474,000 

500,000 

257,000 

261,000 

2,638 

Dec  2001 

1992 

515,000 

500,000 

271,000 

261,000 

2,573 

Dec  2002 

1992 

503,000 

500,000 

267,000 

261,000 

2,803 

Dec  2003 

1992 

522,000 

515,000 

292,000 

271,000 

2,842 

Jun  2004 

2003 

616,000 

522,000 

616,000 

345,000 

292,000 

339,000 

2,842 

Dec  2004 

2003 

670,000 

568,000 

616,000 

347,000 

294,000 

339,000 

2,761 

Dec  2005 

2003 

642,000 

544,000 

656,000 

1,624,000 

314,000 

266,000 

339,000 

831,000 

2,970 

Dec  2006 

2003 

631,000 

535,000 

656,000 

1,624,000 

314,000 

266,000 

339,000 

831,000 

3,067 

Dec  2007 

2003 

601,000 

509,000 

656,000 

1,624,000 

303,000 

257,000 

339,000 

831,000 

3,448 

Dec  2009 

2003 

539,000 

457,000 

544,000 

1,624,000 

277,000 

235,000 

277,000 

831,000 

4,227 

Dec  2010 

2003 

503,000 

426,000 

544,000 

1,624,000 

257,000 

218,000 

277,000 

831,000 

4,606 

Dec  2011 

2003 

533,000 

452,000 

544,000 

1,624,000 

255,000 

216,000 

277,000 

831,000 

4,817 

Dec  2012 

2003 

486,000 

412,000 

533,000 

1,624,000 

243,000 

206,000 

255,000 

831,000 

5,745 

A  RAND  Corporation  study  of  the  MIDS-LVT  program  *  provides  useful  insights.  MIDS 
experieneed  signifieant  schedule  slippage  along  the  way  to  production  and  installation.  A  large 
portion  of  the  slippage  was  due  to  extrinsic  factors.  By  the  same  token,  because  MIDS  terminals 
play  a  central  role  in  many  aircraft  and  other  platforms,  the  program’s  delays  adversely  affected 
them.  Nevertheless,  there  was  little  net  effect  on  MIDS  terminal  prices. 

As  the  utility  and  performance  of  MIDS-LVT  became  apparent  and  as  the  costs  remained 
under  control,  the  market  for  the  terminals  expanded  steadily,  more  than  doubling  in  the  past 
decade,  bringing  further  unit  cost  decreases. 

Although  there  was  a  cost-control  success  one  cannot  unequivocally  associate  it  with 
CAIV.  The  SARs  make  no  mention  of  CAIV  and  give  no  hint  of  any  of  the  cost-performance 
tradeoffs  that  are  the  key  point  of  CAIV.  Thus  it  appears  that  the  program’s  favorable  cost 
performance  is  more  the  product  of  realistic  early  planning,  good  management,  and  the  absence 
of  any  particular  misfortune. 

F.  LCS 

The  LCS  was  not  a  CAIV  flagship  program  but  CAIV  was  highly  central  to  its  acquisition 
strategy,  at  least  declaratively.  Its  acquisition  is  detailed  in  P-4531  and  the  resulting  insights  will 
be  brought  into  the  present  discussion. 

Evaluating  the  effectiveness  of  CAIV  in  the  case  of  the  LCS  presents  some  challenges. 
Data  on  PAUC  and  APUC  are  shown  in  Table  A-3.  The  trends  appear  to  be  favorable,  with 
PAUC  and  APUC  below  the  targets  set  in  the  APB  approved  in  April  2011,  and  falling. 


Table  A-3.  PAUC  and  APUC  for  LCS 
PAUC  (BY2010  $M)  APUC  (BY2010$M) 


B/L  4- 

B/L  4-  Curr.  11  Curr. 


SAR  Date 

11  APB 

Qty. 

Est. 

Qty. 

APB 

Qty. 

Est. 

Qty. 

31  Dec  2010 

582.0 

55 

557.8 

55 

535.3 

53 

511.0 

53 

31  Dec  2011 

582.0 

55 

557.8 

55 

535.3 

53 

535.2 

53 

31  Dec  2012 

582.0 

55 

534.5 

55 

535.3 

53 

485.3 

50 

Yet  the  Milestone  B  decision  that  preceded  the  APB  took  place  seven  years  after  the  first 
ship  contracts  were  awarded  under  research,  development,  test,  and  evaluation  (RDT&E) 
funding.  At  the  time  of  the  initial  contract  award  the  USN  and  the  contractors  were  estimating 
unit  costs  of  less  than  $230  million  (M)  in  EY2005  dollars  ($255M  in  the  BY2010  dollars  of 


Daniel  Gonzales,  Daniel  Norton,  and  Myron  Hura,  Multifunctional  Information  Distribution  System  (MIDS): 
Program  Case  Study,  DB-292-AF  (Santa  Monica,  CA:  RAND  Project  AIR  FORCE,  2000). 
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Table  A-3).  Clearly  the  eurrent  estimate  of  $485. 3M  APUC  for  fifty  ships  represents  substantial 
growth  over  the  2004  estimates. 

RDT&E  and  total  proeurement  eost  estimates  are  shown  in  Table  A-4. 


Table  A-4.  LCS  RDT&E  and  Procurement  Totals 


RDT&E  (2  prototype  ships): 

BY2004  $M 

SAR  B/L  . 
plan  est. 

Current  APB  Concept 

Current 

SAR  date 

Objective 

Threshold 

Est. 

QtY. 

30Jun  2004 

1,172.7 

1,172.7 

1,290.0 

1,172.7 

2 

31  Dec  2004 

1,172.7 

1,172.7 

1,290.0 

1,172.7 

2 

31  Dec  2005 

1,172.7 

1,172.7 

1,290.0 

1,595.9 

2 

31  Dec  2006 

1,172.7 

1,172.7 

1,290.0 

1,791.8 

2 

31  Dec  2007 

1,172.7 

1,172.7 

1,290.0 

2,595.2 

2 

31  Dec  2009 

1,172.7 

1,172.7 

1,290.0 

3,349.4 

2 

BY2010  $M 

31  Dec  2010 

1,339.9 

3,433.3 

3,776.6 

3,426.8 

2 

31  Dec  2011 

3,433.3 

3,433.3 

3,776.6 

3,391.4 

2 

31  Dec  2012 

3,433.3 

3,433.3 

3,776.6 

3,329.1 

2 

Procurement  (53  ships  in  APB): 

BY2010  $M 

QtYv 

Current 

SAR  B/L  . 
plan  est. 

Curr  APB  Concept 

Current 

SAR  date 

Objective 

Threshold 

Est. 

Est. 

31  Dec  2010 

28,369.2 

31,206.1 

28,364.0 

53 

31  Dec  2011 

28,369.2 

28,369.2 

31,206.1 

27,083.4 

53 

31  Dec  2012 

28,369.2 

28,369.2 

31,206.1 

24,266.9 

50 

These  data  show  that  the  RDT&E  (the  great  majority  of  whieh  went  for  the  two  prototype 
ships)  grew  by  more  than  150  pereent  in  eonstant  dollars  between  2004  and  2012.  All  of  the 
growth  took  plaee  in  the  first  five  years  of  this  period  and  have  leveled  off  sinee  then. 

Clearly,  this  does  not  re  fleet  favorably  on  the  applieation  of  CAfV  in  ECS.  The  program 
history  shows  that  in  faet  there  was  a  good  deal  of  effort  to  trade  off  requirements  for  eost 
savings  up  to  the  point  at  whieh  the  basie  ship  designs  were  frozen  late  in  2004.  However,  mueh 
of  this  effort  refleeted  a  signifieant  misunderstanding  of  aetual  eost  impaets.  In  partieular,  it 
appears  that  those  who  were  ehiefly  responsible  for  the  trades  failed  to  reeognize  the  full  eost 
implieations  (for  operating  and  support  (O&S)  as  well  as  aequisition)  of  ship  speed.  By  insisting 
on  a  very  high  ship  speed  they  ensured  high  eosts  despite  serious  saerifiees  in  areas  ineluding 
seakeeping,  enduranee,  survivability,  supportability,  maintainability,  and  durability — many  of 
whieh  will  bring  penalties  in  the  overall  eost  of  ownership.  The  lesson  is  that  effeetive  CAIV 


Believed  required  for  effectiveness  and  survivability  in  littoral  areas. 
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application  demands  a  thorough  understanding  of  the  engineering  as  well  as  operational  faetors, 
and  informed  ealeulation  of  the  effeets  of  speeifie  trades  rather  than  gross  judgment. 

Onee  the  ship  designs  were  largely  frozen  there  was  little  seope  for  any  further  savings 
through  performanee  tradeoffs.  Indeed,  onee  it  beeame  elear  that  light  ship  weight  would 
substantially  exceed  2,000  tons,  as  it  did  by  the  spring  of  2004,  it  was  obvious  that  unless 
something  quite  radieal  was  done  in  terms  of  design  and  eonstruetion,  the  eost  would 
eonsiderably  exeeed  the  $220  million  target.^  At  the  time,  the  Navy  was  thinking  in  terms  of 
designing  and  building  the  LCS  like  a  eommereial  ferry  and  simply  setting  up  mission  equipment 
in  the  spaee  used  for  vehieles — ^whieh  might  have  been  possible  for  not  too  mueh  more  than 
$220  million,  leaving  aside  the  eost  of  the  mission  systems.  However  the  mission  and  operating 
environment  expected  for  the  LCS  in  no  way  resembled  that  of  a  ferry  operating  on  short  routes 
and  only  in  favorable  weather,  with  no  threats  beyond  the  ordinary  hazards  of  the  sea.  As  it 
gradually  was  reeognized  that  a  light  eommereial  vessel  eould  not  be  expeeted  to  serve 
satisfaetorily  as  a  warship,  or  eertainly  not  for  very  long,  the  ships  grew  and  so  did  the  eost  per 
ton.  In  summary,  that  Navy’s  CAIV-like  early  efforts  to  trade  performanee  and  eost  proved  to  be 
highly  unrealistie  and  subsequently  required  major  reversals. 

The  Navy  also  eounted  on  savings  through  a  very  eompressed  sehedule  and  buying  on 
essentially  eommereial  terms,  with  little  government  management  or  supervision.  This  aetually 
inereased  eosts  sinee  the  lack  of  planning  led  to  mueh  work  being  redone  or  done  out  of  the  most 
effieient  sequenee. 

A  summary  of  the  findings  of  IDA  Paper  P-4531  regarding  eost  growth  in  the  LCS  program 
is  in  Figure  A-5."^ 


As  a  general  rule,  ships  in  the  LCS’s  size  range  built  to  light  warship  standards  cost  $175,000  to  $225,000  per 
light  ton,  while  fast  commercial  ships  generally  cost  $100,000  to  $150,000  per  light  ton,  depending  on  the 
quality  and  extent  of  the  accommodations. 

Porter  et  ah.  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  Vol.  Ill,  I-23-I-24. 
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•  Most  of  the  LCS  cost  growth  can  ultimately  be  attributed  to  lack  of  strong 
acquisition  leadership  and  communication.  The  CNO  and  many  others  saw  LCS  as 
a  brand  new  concept  for  warships.  The  concept  envisioned  adapting  a 
commercial  design  to  suit  naval  missions  and  in  doing  so  producing  it  at  lower 
cost  and  much  more  quickly  than  typical  warships. 

•  But  other  groups  within  the  Navy  successfully  imposed  operational  and  technical 
requirements  more  consistent  with  warships  than  commercial  vessels. 

•  The  ultimate  result  was  an  RFP  and  acquisition  strategy  that  were  only  feasible 
under  radically  changed  acquisition  practices  and  wholesale  waivers  of 
traditional  ship  engineering  and  design  practices. 

•  For  one  of  the  two  competing  prototype  contractors,  cost  growth  resulted  from 
poor  performance  largely  due  to  an  inability  to  train  and  maintain  a  skilled 
workforce,  exacerbated  by  Flurricane  Katrina.  In  addition,  the  Austal  shipyard  did 
not  have  rigorous  work  processes,  especially  related  to  systems  engineering. 


Note:  CNO  =  Chief  of  Naval  Operations;  RFP  =  request  for  proposal. 

Figure  A-5.  Reasons  for  Cost  Growth  in  the  LCS  Program  identified  in  iDA  Paper  P-4531 

On  the  whole  the  cost-reduction  efforts  made  by  the  Navy  on  the  LCS  appear  to  have  been 
sincere  and  determined — and  seriously  misguided.  A  much  more  conventional  approach  would 
probably  have  resulted  in  a  product  that  better  fit  operational  needs  at  lower  overall  LCC. 

G.  San  Antonio  Class  Amphibious  Transport  Dock  (LPD  17) 

The  LPD  17  class  was  designed  as  a  replacement  for  several  earlier  types  of  amphibious 
ships.  Although  amphibious  ships  had  once  been  comparatively  simple,  the  LPD  17  incorporates 
a  sophisticated  command  and  control  suite,  as  well  as  a  variety  of  features  intended  to  increase 
efficiency  and  throughput,  putting  it  on  a  plane  of  complexity  close  to  that  of  surface 
combatants. 

The  LPD  17  class  was  not  a  CAIV  flagship  program  but  was  a  CAIV  program  and  was 
planned  to  place  strong  emphasis  on  limiting  cost  growth.  As  laid  out  in  P-4531,  Hurricanes 
Katrina  (2005),  Ike,  and  Gustav  (both  2008)  had  an  effect  on  program  costs  due  to  shipyard 
damage  and  impacts  on  shipyard  operations,  but  poor  planning  and  execution  had  a  greater 
impact.^  The  ship  incorporated  novel  systems  and  components  and  was  designed  with  novel 
techniques  for  construction  with  novel  processes  in  efforts  to  reduce  costs.  In  many  cases  these 
novelties  were  unproven  and/or  unfamiliar  to  the  builder,  and  many  backfired  and  resulted  in 
added  costs. 

Table  A-5  reviews  the  history  of  PAUC  and  APUC  for  the  LPD  17  class.  It  is  evident  that 
the  greater  portion  of  cost  growth  preceded  the  damage  from  Hurricane  Katrina  in  2005. 


Porter  et  al.,  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  Vol.  Ill,  Appendix  A. 
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Table  A-5.  PAUC  and  APUC  for  LPD  17  Class 


SAR 

Date 

PAUC  (BY96  $M) 

APUC  (BY96  $M) 

Quantity 

Curr. 

Est. 

Curr. 

B/L 

Curr. 

Est 

Curr. 

B/L 

Curr. 

B/L 

Dec  1997 

729.2 

751.6 

720.8 

743.8 

12 

12 

Dec  1998 

727.7 

751.6 

719.5 

743.8 

12 

12 

Sep  1999 

727.7 

751.6 

719.5 

743.8 

12 

12 

Dec  1999 

807.8 

751.6 

799.7 

743.8 

12 

12 

Sep  2001 

974.6 

751.6 

962.4 

743.8 

12 

12 

Dec  2001 

1,078.3 

751.6 

1,070.2 

743.8 

12 

12 

Dec  2002 

1,116.6 

1,078.3 

1,108.3 

1,070.2 

12 

12 

Dec  2003 

1,098.7 

1,078.3 

1,090.2 

1,070.2 

12 

12 

Dec  2004 

1,145.0 

1,078.3 

1,132.5 

1,070.2 

9 

12 

Jun  2005 

1,145.0 

1,079.6 

1,132.5 

1,070.2 

9 

12 

Dec  2005 

1,156.8 

1,079.6 

1,144.4 

1,070.2 

9 

12 

Dec  2006 

1,233.7 

1,079.6 

1,221.3 

1,070.2 

9 

12 

Dec  2007 

1,278.7 

1,233.7 

1,265.8 

1,221.3 

9 

9 

Dec  2009 

1,304.5 

1,233.7 

1,294.0 

1,221.3 

11 

9 

Dec  2010 

1,307.2 

1,314.4 

1,296.6 

1,304.3 

11 

11 

Dec  2011 

1,297.2 

1,314.4 

1,286.7 

1,304.3 

11 

11 

Dec  2012 

1,292.8 

1,314.4 

1,282.2 

1,304.3 

11 

11 

A  summary  of  the  findings  in  P-4531  regarding  eost  growth  in  the  LPD- 17  program  is  in 
Figure  A-6.^ 


Porter  et  al.,  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  Vol.  Ill,  A-17-A-18. 
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•  The  Navy  made  optimistic  assumptions  about  factors  driving  costs.  It  is  likely 
that  this  was  due  to  budget-constrained  costing.  Among  other  matters,  the  Navy 
estimated  that  teaming  arrangements  would  cut  costs,  contrary  to  the  usual 
experience  of  cost  increases  associated  with  teaming. 

•  The  Navy  and  CAIG  both  underestimated  the  complexity  of  the  ship. 

•  The  DAE  accepted  the  optimistic  Service  Cost  Position,  evidence  of  its 
inadequacy  notwithstanding. 

•  The  Navy  failed  to  scale  up  program  management  and  oversight  to  mitigate 
risks  associated  with  its  selection  of  a  shipyard  with  no  experience  in  building 
comparably  complex  ships. 

•  The  contractor,  wrestling  with  a  program  of  unfamiliar  complexity,  performed 
poorly.  It  depended  on  a  new  and  untested  system  of  design  tools,  which 
resulted  in  slow  release  of  engineering  drawings.  But  it  moved  ahead  with 
construction  to  meet  schedules,  resulting  in  out-of-sequence  work.  And  it  failed 
to  build  and  maintain  a  workforce  of  the  quality  needed,  resulting  in  serious 
quality  issues. 

•  The  program  was  stretched  out,  due  mostly  to  poor  contractor  performance,  but 
Congressional  and  DoD  program  actions  also  played  some  part. 

•  Hurricane  Katrina  disrupted  work  for  a  considerable  period. 


Figure  A-6.  Reasons  for  LPD-17  Cost  Growth  from  IDA  Paper  P-4531 


There  has  been  no  substantial  requirements  growth  over  the  course  of  the  program.  Prior  to 
finalization  of  the  ship  design,  the  USN  traded  away  defensive  features  for  the  sake  of  reduced 
costs  (provoking  sharp  Congressional  criticism),  but  there  is  not  a  great  deal  left  to  trade  without 
cutting  into  fundamental  safety  and  operability,  or  impacting  USMC  missions. 

There  was  no  clear  reason  to  regard  the  LPD  17  as  ill-suited  to  CAIV  at  the  outset,  but 
overall  it  has  been  quite  unsatisfactory  from  a  CAIV  standpoint. 


H.  EFV 

The  U.  S.  Marine  Corps’  EFV  was  conceived  to  be  an  infantry  fighting  vehicle  capable  of 
high  speed  ashore  and  also  very  high-speed  afloat  in  its  amphibious  mode.  Its  story  is  told  in  an 
earlier  IDA  paper  and  will  not  be  repeated  here.^  At  the  time  that  publication  was  completed  in 
2009,  the  program  had  been  restarted  and  returned  to  the  System  Development  and 

o 

Demonstration  (SDD)  phase,  after  operational  test  vehicles  proved  to  be  unable  to  complete 
operational  test  and  evaluation  (OT&E)  due  to  very  severe  reliability  deficiencies.  However, 
EFV  was  deleted  from  the  FY2012  President’s  Budget  request  and  on  16  Febmary  2011  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  (USD(AT&L))  signed  an 
acquisition  decision  memorandum  (ADM)  directing  orderly  cancellation. 


Porter  et  al.,  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  Vol.  Ill,  Appendix  C. 
Now  referred  to  as  Engineering  and  Manufacturing  Development  (EMD). 
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In  announcing  the  cancellation,  the  Commandant  of  the  Marine  Corps  (CMC)  said, 
“Despite  the  eritieal  amphibious  and  warfighting  eapability  the  EFV  represents,  the  program  is 
simply  not  affordable  given  likely  USMC  proeurement  budgets,”  going  on  to  say  that  he  had 
decided  to  pursue  a  more  affordable  vehicle,  named  the  Amphibious  Combat  Vehiele  (ACV).^ 
The  deeision  to  eaneel  has  been  sharply  eritieized  in  Congress  and  elsewhere  and  Congress  has 
direeted  that  the  EFV  be  eonsidered  among  the  options  for  ACV.  The  status  of  the  ACV  is 
uneertain,  but  in  any  event  it  is  not  organized  or  managed  as  a  eontinuation  of  the  EFV  program. 

Eeaving  aside  the  anomalous  figures  for  Deeember  2010  resulting  from  the  deeision  to 
defund  produetion.  Table  A-6  shows  that  the  unit  proeurement  eost  more  than  tripled  in  the  six 
years  following  program  initiation.  Sinee  the  eost  was  high  for  eombat  vehieles  to  begin  with, 
it  is  easy  to  see  how  it  led  to  a  judgment  that  the  program  was  not  affordable;  what  seems 
remarkable  is  that  it  took  another  three  years  to  aet  on  it  and  eaneel  the  program. 


Table  A-6.  PAUC  and  APUC  for  EFV 


SAR 

Date 

BY1993  $M 

BY2007  $M 

Prodn 

Qty 

PAUC 

APUC 

PAUC 

APUC 

Sep  2001 

6.5 

5.3 

1013 

Dec  2001 

7.1 

5.7 

1013 

Dec  2002 

7.9 

6.3 

1013 

Dec  2003 

7.9 

6.2 

1013 

Dec  2004 

8.8 

6.9 

1013 

Dec  2005 

8.7 

6.8 

1013 

Dec  2006 

13.8 

9.9 

573 

Jun  2007 

17.4 

13.1 

22.2 

16.8 

574 

Dec  2007 

17.4 

13.1 

22.2 

16.7 

574 

Dec  2009 

17.5 

13.1 

22.3 

16.8 

574 

Dec  2010 

134.6 

158.5 

172.1 

202.8 

1 

A  summary  of  the  findings  from  IDA  paper  P-4531  regarding  eost  growth  in  the  EFV 
program  is  in  Figure  A-7.^^ 


Andrew  Feickert,  “Marine  Corps  Amphibious  Combat  Vehiele  (ACV)  and  Marine  Personnel  Carrier  (MPC): 
Background  and  Issues  for  Congress.” 

The  reduction  in  quantity  in  2006  was  not  a  major  cause  of  cost  increases;  see  Porter  et  al.,  The  Major  Causes  of 
Cost  Growth  in  Defense  Acquisition,  Vol.  Ill,  C-6-C-7. 

Porter  et  al.,  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  Vol.  Ill,  C-I4-C-I5. 
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The  most  fundamental  cause  of  the  EFV's  cost  growth  was  persistent  failure  to  recognize  and  address 

the  real  nature  and  challenges  of  the  technical  problem  presented  by  the  requirements.  This  led  to  a 

series  of  serious  deficiencies: 

*  Inadequate  funding 

The  EMD,  in  particular,  was  funded  at  levels  which  made  effective  execution  impossible. 

*  Unrealistic  schedules 

Even  though  the  schedule  was  repeatedly  extended,  the  plan  never  caught  up  with  the  reality  of 
what  had  to  be  done. 

*  Excessive  concurrency  and  compression 

To  meet  the  unrealistic  schedule,  too  much  concurrency  and  compression  was  attempted.  The 
critical  design  review  was  held  within  a  month  of  EMD  contract  award  and  before  analyzing  the 
results  of  development  testing-one  example  of  many  resulting  in  the  need  for  extensive  re-work. 

*  Skipping  critical  tasks 

Another  impact  of  attempting  too  much  with  too  little  was  the  lack  of  focus  on  some  critical  tasks, 
most  notably  systems  and  reliability  engineering.  The  result  was  that  the  program  encountered 
extremely  damaging  and  costly  problems. 

*  Major  underestimation  of  procurement  unit  costs 

Because  the  nature  of  the  system  and  its  challenges  were  not  clearly  understood  or  recognized, 
production  costs  were  estimated  on  the  basis  of  supposed  similarity  to  much  simpler  and  less 
powerful  armored  vehicles. 

Figure  A-7.  Reasons  for  EFV  Cost  Growth  from  IDA  Paper  P-4531 

While  EFV  was  not  a  CAIV  flagship  program,  in  1998  it  won  the  Packard  Award  for 
acquisition  excellence  in,  among  other  factors,  CAIV  execution.  The  award  reflected  cost- 
requirements  tradeoffs  made  during  the  PDRR  phase,  when,  of  course,  there  was  more  scope  to 
make  them  than  at  later  dates.  Despite  those  tradeoffs,  the  program  entered  full-scale 
development  with  little  to  trade  off  if  the  basic  requirement  was  to  be  met  using  the  platform 
developed  in  PDRR. 

In  effect,  the  sharp  cost  increase  signaled  that  the  engineering  risks  had  been  inadequately 
recognized  and  hedged  against  prior  to  SDD.  Because  the  engineering  issues  were  not  well 
understood,  the  cost  estimates  founded  on  them  were  faulty.  Thus  the  cost-requirements  tradeoffs 
in  PDRR  were  ultimately  unsuccessful  in  containing  costs. 

If  the  ACV  program  does  go  ahead  with  markedly  reduced  performance  goals,  it  will  be,  in 
effect,  a  CAIV-like  trade  off  of  requirements  to  cut  costs.  Much  money  and  time  could  have  been 
saved  had  realistic  cost-constrained  trade  studies  been  made  prior  to  EFV’s  entry  into  full-scale 
development.  In  other  words,  in  spite  of  the  claims,  a  CAIV  approach  was  not  really  taken. 

I.  F-35  Joint  Strike  Fighter  (F-35) 

The  F-35  program  (previously  called  the  JSF)  is  producing  a  strike  fighter  to  serve  the 
USAF,  USN,  and  USMC,  as  well  as  air  forces  in  at  least  eight  allied  nations.  In  order  to  meet  the 
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needs  of  various  users  it  is  produeed  in  three  distinet  versions.  (See  IDA  Paper  P-4531, 
Appendix  G,  for  a  more  eomplete  discussion  current  through  early  2009.) 

The  F-35  is  a  CAIV  flagship  program  and  affordability  was  stressed  heavily  when  the 
program  was  initially  being  considered  in  1993-95.  A  variety  of  quite  advanced  technologies 
were  incorporated  to  enhance  mission  capabilities  and,  it  was  hoped,  to  reduce  O&S  costs.  As  of 
the  end  of  2013,  eighty- five  aircraft  had  been  delivered  and  more  than  4,000  hours  of  flight  test 
completed,  but  the  program  continues  to  have  significant  development  problems  with  some 
critical  technologies,  especially  software  and  the  Helmet-Mounted  Display  System  (HMDS). 

Table  A-7  briefly  reviews  the  history  of  F-35  unit  costs.  Although  cost  growth  has  leveled 
out  and  even  declined  slightly  since  2010,  costs  remain  substantially  greater  than  originally 
planned.  Because  of  the  size  and  scope  of  the  program,  cost  growth  has  received  especially  wide 
public  and  Congressional  attention.  It  has  been  the  subject  of  several  studies,  including  two  root- 
cause  analyses  for  the  OUSD(AT&L)  Office  of  Performance  Assessments  and  Root  Cause 
Analyses  (PARC A). 


Table  A-7.  PAUC,  APUC,  and  Unit  Recurring  Flyaway  (URF)  Costs  for  F-35 


Source 

Date 

BY2002  $M 

PAUC  APUC  URF 

U.S. 

Proc. 

Qty. 

Sep  1995 

34.1 

2,978 

Dec  1997 

34.9 

2,852 

Dec  1998 

34.9 

2,852 

Dec  1999 

34.9 

2,852 

Dec  2001 

61.9 

50.9 

40.9 

2,852 

Dec  2002 

65.7 

51.6 

41.0 

2,443 

Dec  2003 

78.0 

61.2 

47.4 

2,443 

Dec  2004 

78.3 

61.5 

47.9 

2,443 

Dec  2005 

82.1 

66.0 

50.3 

2,443 

Dec  2006 

85.2 

69.2 

53.0 

2,443 

Dec  2007 

85.5 

69.3 

54.5 

2,443 

Dec  2009 

97.1 

79.0 

60.7 

2,443 

Dec  2010 

110.1 

90.8 

68.2 

2,443 

Dec  2011 

110.2 

89.4 

68.4 

2,443 

Dec  2012 

106.5 

85.9 

65.7 

2,443 

A  variety  of  devices  were  relied  upon  for  cost  control  in  the  F-35  program,  including  early 
competitive  prototyping,  commonality  across  the  three  major  variants,  commercial-type 
acquisition  approach,  cost-oriented  technology  innovations,  cost-oriented  innovations  in 
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development  and  testing,  a  eompressed  schedule  with  steep  production  ramp-up,  and  stable 
program  funding. 

The  initial  program  thrust  and  structure  was  shaped  by  a  desire  to  gain  support  from  many 
quarters  by  promising  something  for  everyone,  prompting  the  combination  of  three  quite 
different  variants  in  one,  a  very  broad  mission  set  and  demanding  performance  requirements  in 
multiple  dimensions,  and  commitments  to  parcel  the  work  out  among  a  great  many  firms  and 
organizations.  There  was  scarcely  any  systems  engineering  to  understand  the  implication  of  all 
these  requirements  and  commitments  before  competitive  concept  definition  (CD)  began  in  the 
fall  of  1996;  the  government  aircraft  systems  engineering  capabilities  had  been  slashed  in  the 
1980s  and  1990s  but  the  aircraft  industry  was  not  well  prepared  to  step  into  the  breach.  The  focus 
of  the  contractors  competing  in  the  CD  phase  was  on  winning  the  all-important  SDD  contract.  As 
a  result,  at  the  end  of  CD  serious  engineering  issues  and  risks  remained  open,  and  not  even 
identified,  like  unexploded  ordnance. 

The  program  office  and  contractor  mechanisms  for  warning  of  these  bombs  were 
inadequate.  This  situation  culminated  in  2003  in  realization  that  the  designs  were  very 
substantially  overweight  and  that  conventional  methods  of  weight  control  were  not  able  to  deal 
with  the  problem.  This  would  have  been  serious  at  best  but  the  inclusion  of  the  short 
takeoff/vertical  landing  (STOVL)  variant  in  the  mix  meant  that  weight  was  absolutely  critical;  if 
the  landing  weight  of  that  variant  exceeded  the  available  lift  thrust  the  aircraft  simply  could  not 
operate.  The  engine  had  already  been  stretched  considerably,  which  meant  that  major  weight 
reduction  was  essential.  The  only  solution  was  to  redesign  the  aircraft  almost  from  scratch,  even 
though  assembly  of  production  prototypes  had  already  begun.  Not  only  was  redesign  expensive 
in  itself  but  in  the  process  it  was  necessary  to  give  up  many  of  the  features  that  had  been  counted 
upon  to  save  money.  Moreover,  even  with  all  possible  weight  wrung  out  of  the  aircraft  it  was 
necessary  to  extract  still  more  power  from  the  engine,  which  further  increased  its  cost.  And  to 
top  it  off,  the  redesign  and  restart  seriously  disrupted  the  tight  schedules  that  had  been  carefully 
optimized  for  cost. 

The  findings  on  the  causes  of  cost  growth  in  the  JSF/F-35  program  (through  early  2009)  are 
12 

shown  in  Figure  A-8. 


Porter  et  al.,  The  Major  Causes  of  Cost  Growth  in  Defense  Acquisition,  Vof  III,  G-15-G-16. 
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STOVL  variant 

The  unappreciated  complexities  of  the  STOVL  variant  led  to  the  2003-2004  design  crisis.  The  inclusion  of 
the  STOVL  added  greatly  to  the  complexity  of  the  whole  program.  Lack  of  relevant  experience  with 
STOVL  aircraft  hindered  estimation  of  weights  and  costs,  and  slowed  recognition  of  problems.  The 
stringent  demands  of  vertical  landing  demanded  more  radical  solutions  to  the  weight  problem  than 
might  have  otherwise  been  necessary.  Including  the  STOVL  requirement  without  a  sound  understanding 
of  STOVL's  true  costs  tilted  the  early  assessments  of  cost-effectiveness  in  favor  of  its  inclusion. 

Inefficient  concurrency  in  development 

Concurrency  was  deliberately  adopted  as  a  strategy  for  meeting  demanding  requirements  for  service 
dates  and  for  limiting  development  cost,  but  the  sequencing  of  activities  within  the  concurrent  schedule 
sharply  increased  risk  relative  to  an  optimum  plan,  and  led  directly  to  substantial  cost  growth,  and 
ultimately  and  ironically,  schedule  delays.  Early  development  efforts  should  have  concentrated  on 
identifying  and  reducing  the  highest  risk  areas,  but  they  were,  in  fact,  spread  over  many  areas  of  lower 
risk,  which  delayed  recognition  of  the  critical  weight  problem  and  led  to  a  great  deal  of  re-work. 

Failure  to  use  very  low  observable  (VLO)  aircraft  weight  data 

The  government's  decision  not  to  allow  use  of  weight  data  from  other  VLO  aircraft  in  the  statistical 
weight  estimation  process  for  JSF  reduced  both  the  accuracy  of  the  weight  estimates  and  the  confidence 
engineers  and  managers  could  place  in  them,  and  contributed  to  setting  the  stage  for  the  weight  crisis. 

Unforeseeable  price  increases 

Materials  price  increases  were  a  significant  source  of  cost  increases  by  the  end  of  2007.  They  reflect 
economic  trends  and  thus  lay  beyond  the  ability  to  forecast  or  influence. 

Figure  A-8.  Reasons  for  F-35  Cost  Growth  from  IDA  Paper  P-4531 

Although  the  discovery  of  the  mistakes  on  weights  and  its  sequelae  was  not  the  only  factor 
in  cost  growth,  it  was  the  major  one.  The  software  and  helmet  problems  (and  earlier  problems 
with  critical  operational  support  systems)  have  increased  development  and  test  costs  and  led  to 
further  delays. 

The  question  remains  as  to  whether  the  CAIV  approach  had  any  real  impact  on  the  JSF/F- 
35  program.  Because  of  the  large  numbers  of  aircraft  that  the  F-35  will  replace,  affordability,  and 
by  inference  procurement  costs,  have  always  had  a  strong  focus  in  the  program.  Without  a 
much  more  in-depth  review,  the  research  team  has  been  unable  to  determine  what  types  of 
performance-versus-cost  tradeoffs  might  have  been  made  along  the  way.  The  big  tradeoff  that 
the  Department  was,  obviously,  not  willing  to  make  was  on  the  STOVL  capability  needed  to 
replace  the  AV-8B  Harriers  of  both  the  USMC  and  the  United  Kingdom. 

J.  EELV 

The  EELV  program  began  in  the  early  1990s  with  the  objective  of  assuring  adequate  launch 
services  for  military  and  intelligence  payloads  at  affordable  prices.  It  has  been  reasonably 
successful  in  technical,  schedule,  and  operational  terms,  but  much  less  so  economically.  Costs 


See  “Joint  Strike  Fighter  Total  Cost  Still  up  in  the  Air,”  National  Defense,  November  2013.  The  program 
manager  was  quoted  as  saying:  “by  2019,  I’m  looking  for  a  fifth-generation  airplane  for  a  fourth-generation 
price.” 
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were  to  have  been  controlled  through  multilevel  competition  and  cost-sharing,  but  the  underlying 
planning  assumptions  have  proven  to  be  erroneous  and  costs  have  grown  by  more  than  300 
percent. 

A  complete  analysis  of  EELV  cost  growth  lies  beyond  the  scope  of  this  paper,  but  it  is 
possible  to  give  a  broad  sketch  of  major  factors. 

The  USAE  has  responsibility  for  launching  all  DOD  and  National  Reconnaissance  Office 
(NRO)  space  payloads.  Its  performance  of  this  mission  has  been  affected  not  only  by  its  own 
policies  and  DOD  acquisition  policy  in  general  but  by  overall  national  policies  regarding  space. 
The  interaction  of  these  policies  in  the  1970s  and  1980s  was  complex  and  destabilizing  for  the 
USAE,  which  in  the  meantime  continued  to  use  1960s-era  expendable  (i.e.,  one-time)  launch 
vehicles  (EV).'"^ 

After  a  decade  of  abortive  programs  beginning  in  the  mid-1980s,  the  USAE  and  DOD 
decided  to  pursue  what  became  the  EEEV.^^  As  the  title  suggests,  the  plan  involved  adapting 
improved,  i.e.,  “evolved,”  versions  of  some  existing  launchers — based  on  the  conclusion  that  the 
payoff  in  reduced  operational  costs  from  developing  any  altogether  new  launcher  system  would 
not  be  sufficient  to  justify  the  higher  initial  cost  and  risk. 

Early  in  the  1980s  the  U.S.  government  had  abandoned  the  policy  requirement  that 
government  organizations  provide  all  launch  services — the  USAE  for  all  military  and 
intelligence  payloads  and  the  National  Aeronautics  and  Space  Administration  (NASA)  for 
commercial  payloads  in  addition  to  NASA’s  own  and  other  government  payloads.  Instead, 
licensed  commercial  launch  providers  were  allowed  and  encouraged  to  develop  or  acquire  their 
own  vehicles  and  capabilities. 

In  the  early  1990s,  many  experts  were  confident  that  the  market  for  space  launches  was  on 
track  for  quite  substantial  growth,  and  EEEV  was  planned  to  use  this  trend  to  reduce  costs  to  the 
government.  (The  EEEV  Program  Office  relied  on  the  forecasts  issued  annually  by  the  Eederal 
Aviation  Administration’s  Office  of  Commercial  Space  Transportation  and  the  Commercial 
Space  Transportation  Advisory  Committee.  )  DOD  would  select  one  of  the  existing  families  of 
launchers  as  the  most  promising  and  upgrade  it  to  DOD  standards.  Then,  in  addition  to 


The  history  is  reviewed  in  Carl  E.  Behrens,  Space  Launch  Vehicles:  Government  Activities,  Commercial 
Competition,  and  Satellite  Exports  (Washington,  DC:  Congressional  Research  Service,  20  March  2006)  and  in 
U.S.  Government  Accounting  Office  (GAO),  Access  to  Space:  Issues  Associated  with  DOD’s  Evolved 
Expendable  Launch  Vehicle  Program,  GAO/NSIAD-97-130  (Washington,  DC:  GAO,  June  1997). 

U.S.  Air  Force,  “DOD  Space  Launch  Modernization  Plan,”  briefing  (Washington,  DC:  USAF,  17  May  1994). 

For  an  unofficial  but  informed  account  of  EELV’s  origins  and  early  history  see  Wayne  Eleazer,  “EELV  or 
Never?”  The  Space  Review  (4  Dec  2006)  http://www.thespacereview.eom/article/758/l. 

Federal  Aviation  Administration,  “Reports  &  Studies:  Commercial  Space  Transportation  Forecasts,” 
http://www.faa.gov/about/office  org/headquarters  offices/ast/reports  studies/forecasts/. 
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producing  launchers  for  DOD,  the  contractor  would  also  market  them  to  commereial  eustomers, 
and  DOD  would  benefit  from  the  redueed  priees  as  produetion  volume  grew. 

By  the  mid-1990s,  doubts  about  future  demand  were  growing.  RussianAJkrainian  and 

1 8 

Chinese  firms  were  marketing  launehers  based  on  Cold  War  missiles  at  low  prices. 
Nevertheless,  the  EELV  program  eontinued  to  press  ahead  and  to  plan  on  growing  commereial 
sales. 


Originally  prompted  by  urging  from  DOD  leaders  and  then  sustained  by  finaneial  market 
enthusiasm,  the  1990s  saw  a  great  consolidation  of  the  defense/aerospace  industry.  With  the 
defense  industry  consolidations  of  the  1990s,  all  existing  launeher  produetion  fell  under  just  two 
firms,  Boeing  and  Eockheed  Martin  (EM). 

Although  the  plan  had  been  to  down-seleet  to  just  one  line  of  launch  vehicles  for  EEEV,  in 
1997  it  was  decided  to  select  two,  one  for  each  contractor.  As  explained  in  the  initial  SAR 

A  new  aequisition  strategy  for  EEEV  was  approved  by  USD(A&T)  on  November 
3,  1997.  The  previous  strategy  was  to  award  a  development  eontraet  and  a  launeh 
serviees  eontraet  to  only  one  EEEV  eontraetor.  The  new  approaeh  allows  two 
contraetors  to  enter  the  Engineering  and  Manufaeturing  Development 
(EMD)/Initial  Eaunch  Services  (lES)  phase.  The  strategy  also  maintains 
competition  throughout  the  life  of  the  program,  leverages  the  growing  commereial 
launch  market,  caps  the  Government’s  EMD  costs,  allows  partnering  with 
industry,  while  still  reducing  the  Government's  overall  eost  to  launeh  the  National 
Mission  Model  (NMM)  by  at  least  25%  over  existing  systems. 

Note  that  this  deeision  plans  to  aehieve  eost  avoidanee,  in  aceordanee  with  the  principles  of 
CAIV,  almost  entirely  via  competition  between  the  two  contraetors,  and  with  other  providers.  As 
the  then-program  manager  explained  in  2002, 

DOD  benefited  from  the  opportunity  to  retain  two  proven  launch  service 
providers  for  less  than  the  priee  of  one,  eaptured  over  $500  million  in  immediate 
development  eost  savings,  and  leveraged  the  commereial  satellite  market  to 
reduee  overall  program  risk.  Industry  would  now  fund  the  additional  $3  billion 
required  to  bring  both  systems  to  market;  and  if  market  eonditions  turned  and  one 
EEEV  provider  exited,  DOD  would  be  no  worse  off  than  if  it  had  stayed  with  the 
original  down-select  strategy.  DOD  also  benefited  signifieantly  from  the 
eommereial  satellite  industry’s  mounting  demand  for  launch  services.  Viewed  as 
EEEV’s  silent  “third  partner,”  the  satellite  industry’s  demand  for  reliable  spaee 
lift  and  willingness  to  “jump”  from  one  launeh  service  provider  to  another  in  the 
event  of  a  launeh  failure  have  made  reliability  a  program  touehstone.^^ 


John  Burgess,  “American  Industry  Is  Falling  Behind  in  the  Space  Race,”  New  York  Times,  22  Feb  1994. 
SAR  as  of  31  Dec  1997,  5. 

R.  K.  Saxer  et  ah,  “Evolved  Expendable  Launch  Vehicle  System:  The  Next  Step  in  Affordable  Space 
Transportation,”  Program  Manager  31,  no.  2  (March-April  2002):  2-15,  6. 
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The  government  paid  a  total  of  $1  billion  toward  development  of  the  new  LVs,  with 
Loekheed  Martin  investing  $1.6  billion  and  Boeing  $2.3  billion.^^  Of  eourse  the  eontraetors  have 
reeovered  a  substantial  portion  of  this  investment  through  overhead  eharges  on  government 
business,  but  it,  nevertheless,  is  a  strong  indication  of  the  confidence  they  then  felt  in  the 
commercial  market  prospects. 

Development  of  the  evolved  Atlas  (LM)  and  Delta  (Boeing)  proceeded  without  unusual 
problems  and  the  LVs  met  their  performance  goals.  There  were  no  test  flights — all  launches  have 
carried  actual  payloads — and  this  has  prevented  formal  demonstration  of  performance,  but  the 
in-service  reliability  record  of  the  LVs  has  been  very  favorable.  Thus  the  program  has  succeeded 
in  providing  reliable  operational  launch  capabilities,  although  no  more  reliable  than  those 
provided  by  some  commercial  services  such  as  Europe’s  Arianespace. 

In  explaining  the  program  strategy  in  a  2002  article,  the  program  manager  for  EELV 
stressed  CAIV  and  its  benefits: 

The  single  most  important  tool  within  the  EEEV  “system  of  systems”  design 
trade-space  is  Cost  As  an  Independent  Variable  (CAIV).  Both  EEEV  contractors 
have  made  CAIV  an  integral  part  of  all  EEEV  system  design,  development, 
production,  and  operations  activities  since  program  inception.  CAIV  is  a  powerful 
tool,  providing  for  the  establishment  of  aggressive,  realistic  cost  objectives  and 
the  equally  aggressive  management  of  all  associated  risks.  The  emphasis  on 
CAIV  is  the  major  reason  why  EEEV  has  been  able  to  achieve  its  substantial  life 
cycle  cost-reduction  goals  and  better  position  the  U.S.  commercial  launch 
industry  to  be  more  competitive  in  the  international  marketplace.^^ 

Despite  this  impressive  endorsement  of  CAIV,  at  that  time  the  program  was  collecting  and 
evaluating  little  data  on  contractor  performance.  In  particular,  it  did  not  require  the  contractors  to 
maintain  a  government  certified  earned  value  management  (EVM)  system,  relying  instead  on 
their  own  internal  management  systems.  The  program  office  had  access  to  the  relevant  contractor 
internal  data,  but  lacked  the  personnel  to  adequately  keep  track  of  it  or  to  analyze  its 
implications.  Thus  it  is  clear  that  the  program  was  relying  primarily  on  competitive  pressures 
for  cost  control.  This  was  consistent  with  the  general  thrust  of  the  acquisition  reform  policies  that 
were  instituted  in  the  1990s,  and  particularly  as  they  were  applied  to  National  Security  Space 
(NSS)  programs. 


Congressionally  Mandated  National  Security  Space  Launch  Requirements  Panel,  National  Security  Space 
Launch  Report,  MG-503-OSD  (Santa  Monica,  CA:  RAND  National  Defense  Research  Institute,  16  Aug  2006), 
28-29. 

Saxer  et  al.,  “Evolved  Expendable  Launch  Vehicle  System,”  4. 

U.S.  Government  Accountability  Office  (GAO),  Space  Acquisitions:  Uncertainties  in  the  Evolved  Expendable 
Launch  Vehicle  Program  Pose  Management  and  Oversight  Challenges,  GAO-08-1039  (Washington,  DC: 
GAO,  Sep  2008),  23. 
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Figure  A-9  shows  how  hopes  of  major  eost  sharing  through  commereial  launch  business 
were  doomed  by  market  forces.  This  chart  refers  specifically  to  the  economically  dominant  GEO 
market  segment  but  trends  in  the  low-earth  orbit  (LEO)  segment  were  broadly  similar.  The  boom 
that  had  been  so  confidently  forecast  in  the  latter  half  of  the  1990s  had  prompted  a  great 
expansion  of  worldwide  launch  capacity,  with  many  providers  benefitting  from  subsidies  from 
governments  eager  to  enter  or  remain  in  this  prestigious  and  strategically  critical  business.  But 
the  boom  itself  failed  to  materialize,  leaving  a  large  and  widening  gap  between  demand  and 
supply,  leading  naturally  to  falling  prices.  Although  it  could  not  be  fully  commoditized,  space 
launch  became  much  more  competitive,  and  both  EM  and  Boeing  were  at  a  disadvantage  relative 
to  foreign,  state-subsidized  competitors,  and  their  DOD  partner  with  them. 
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SOURCE:  David  Cavossa,  Satellite  Industries  Association,  "'State  of  the  Satellite  Industry'  Report,"  briefing  to  the 
Panel,  September  14,  2005. 
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Figure  A-9.  A  Growing  Gap  between  Demand  and  Suppiy  for  Space  Launches 


Of  course  if  improved  versions  of  the  Atlas  and  Delta  LVs  had  not  been  developed  under 
EELV,  the  legacy  designs  would  have  been  that  much  less  competitive  than  the  new  versions.  In 
May  2004  the  EELV  Program  Office  estimated  that  the  program,  with  all  its  problems,  had, 
nevertheless,  saved  over  50  percent  relative  to  retaining  the  heritage  LVs  in  service.  In  its  review 
of  the  program,  the  Government  Accountability  Office  (GAO)  noted  that  it  was  unable  to  verify 

24 

these  savings,  but  did  not  directly  challenge  them. 


GAO,  Defense  Space  Activities:  Continuation  of  Evolved  Expendable  Launch  Vehicle  Program ’s  Progress  to 
Date  Subject  to  Some  Uncertainty,  GAO-04-778R  (Washington,  DC:  GAO,  24  Jun  2004),  5. 
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In  any  event,  in  2004  the  program  recognized  large  increases  in  PAUC  and  APUC 
compared  with  the  APB,  as  can  be  seen  in  Figure  A- 10.  The  lack  of  detailed  cost  reporting  made 
it  more  difficult  to  be  certain  of  the  origins  of  this  growth,  but  the  program  office  attributed  well 
over  half  of  it  to  lack  of  commercial  sales.  Other  significant  factors  (mostly  outside  the  control  of 
the  program  office)  included  satellite  weight  growth  requiring  the  use  of  larger  LV  variants  as 
well  as  steps  that  had  been  taken  to  improve  assurance  of  successful  launch  in  light  of  a  series  of 
costly  failures  of  earlier  TVs  in  the  late  1990s.  The  cost  growth  occasioned  a  Nunn-McCurdy 

25 

breach  report,  with  recertification  completed  on  26  April  2004. 
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Figure  A-10.  EELV  Unit  Cost  History 


Another  factor  that  increased  costs  and  was  a  great  complication  in  the  program  was  the 
2003  discovery  that  Boeing  had  42,000  pages  of  Lockheed  Martin  documents  on  the  Atlas  V, 
obtained  through  industrial  espionage,  allegedly  with  the  collusion  of  USAF  officials.  The 
government  suspended  Boeing  from  bidding  on  launcher  contracts — ^but  then  made  exceptions. 
LM  sued  and  it  was  claimed  that  the  legal  battles  could  severely  disrupt  LV  procurement. 

In  2005,  while  the  legal  struggle  was  in  progress,  Boeing  and  Lockheed  Martin  agreed  to 
form  a  joint  venture,  to  be  known  as  United  Launch  Alliance  (ULA)  that  would  pool  all  the 

Ibid.,  7-9. 
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Peter  Pae,  “Boeing's  Misconduct  is  Detailed  in  Memo,”  Los  Angeles  Times,  April  23,  2004,  C.l. 
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operations  to  support  the  EELV  program.  (Commereial  business,  of  whieh  there  eontinues  to  be 
very  little,  remained  the  responsibility  of  the  parent  eompanies.)  The  firms  and  DOD  said  they 
envisioned  substantial  savings  from  consolidation,  but  the  Eederal  Trade  Commission  (ETC) 
opposed  the  arrangement,  seeing  it  as  a  monopoly  that  would  have  the  usual  defects  of 
monopolies.  DOD  said  it  was  justified  by  military  necessity  and  the  ETC  gave  way.  As  part  of 
the  deal,  EM  dropped  its  legal  action  against  Boeing.  The  new  venture  became  operational  in  late 
2006.^’ 

As  with  JASSM  and  SBIRS,  the  USAE  pursued  acquisition  reform  with  particular  vigor  in 
NSS  programs,  and  the  results  have  all  been  disappointing.  Eate  in  2004  a  revision  to  the 
Guidance  for  DOD  Space  System  Acquisition  Process  substantially  tightened  the  process. 
However,  in  negotiating  the  2005  contracts  requirements  for  certified  cost  or  pricing  data  were 
waived.  (This  appears  to  have  been  based  on  confidence  that  the  program  office  understood  the 
contractors’  cost  structure  adequately  without  such  data.)  In  any  event,  the  program  office  lacked 
the  resources  to  adequately  review  and  analyze  any  additional  cost  information.  Efforts  to 
strengthen  this  capability  were  stymied  by  the  effects  of  Service-wide  personnel  cuts  intended  to 
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reduce  overall  USAE  costs. 

In  August  2007  the  EEEV  program  was  removed  from  the  MDAP  category  I  and 
transitioned  to  “sustainment  status.”  Office  of  the  Secretary  of  Defense  (OSD)  officials  told 
GAO  auditors  that  the  move  was  impelled  by  further  cost  growth  that  would  soon  have  required 
a  Nunn-McCurdy  breach  report.  By  moving  to  sustainment  status  the  need  to  report  the  breach 
and  go  through  the  recertification  process  was  avoided.  Moreover,  it  appears  that  EEEV  received 
a  waiver  from  the  requirement  to  submit  EVM  and  other  cost  and  performance  reporting,  thus 
reducing  contract  costs  by  eliminating  the  effort  required  to  gather  and  report  the  data. 
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Renae  Merle,  “Rocket  Monopoly  Approved;  Boeing-Lockheed  Alliance  Likely  to  Increase  Costs,”  Washington 
Post,  4  Oct  2006,  D.l. 

DOD,  Guidance  for  DOD  Space  System  Acquisition  Process.  National  Security  Space  Acquisition  Policy  No. 
03-01  (Washington,  DC:  DOD,  27  Dec  2004). 

GAO,  Evolved  Expendable  Launch  Vehicle:  DOD  Is  Addressing  Knowledge  Gaps  in  Its  New  Acquisition 
Strategy,  GAO-12-822  (Washington,  DC:  GAO,  13  Aug  2012),  4. 

GAO  report  Space  Acquisitions:  Uncertainties  in  the  Evolved  Expendable  Launch  Vehicle  Program  Pose 
Management  and  Oversight  Challenges  states,  “In  addition,  the  EELV  program  office  believes  a  shortage  of 
staff  with  the  right  skills  and  experience  is  limiting  its  ability  to  effectively  manage  the  program.  Air  Force 
staffing  initiatives  have  done  little  to  alleviate  this  shortage.”  GAO,  Space  Acquisitions,  GAO-08-1039,  23-32. 

The  executive  summary  of  the  March  2007  exception  SAR  states:  “The  approved  APB  places  the  EELV 
program  into  sustainment  and  therefore  EELV  is  no  longer  an  active  Major  Defense  Acquisition  Program 
(MDAP). This  will  be  the  final  SAR  submission  for  this  program.”  The  requirement  for  SAR  reporting  is  in 
section  2432  of  Title  10  of  the  U.S.  Code.  According  to  that  section,  SAR  reporting  only  terminates  when  90 
percent  of  the  deliveries  have  been  made  or  90  percent  of  the  planned  funds  spent.  The  March  2007  SAR 
reports  that  only  6  percent  of  deliveries  had  occurred  and  16  percent  of  planned  funds  spent. 

GAO  report  Space  Acquisitions:  Uncertainties  in  the  Evolved  Expendable  Launch  Vehicle  Program  Pose 
Management  and  Oversight  Challenges  states,  “However,  according  to  OSD  officials,  the  sustainment  decision 
may  have  been  influenced  by  other  factors  such  as  avoiding  imminent  Nunn-McCurdy  unit  cost  breaches  and 
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At  this  point  the  competitive  strategies  that  DOD  had  relied  upon  to  control  EELV  costs 
were  in  ruins.  In  principle  Boeing  and  EM  were  still  offering  launch  services  to  commercial 
customers,  but  neither  firm  was  actively  marketing  them  and  few  customers  were  seeking  them. 
Indeed,  Boeing  was  the  largest  single  partner  in  a  firm  offering  launch  services  using 
RussianAJkrainian  EVs  launched  from  a  sea-based  platform  that  competed  with  its  own  Delta 
EVs. 

With  little  prospect  of  commercial  business,  Boeing  and  EM  had  no  incentive  to  compete 
with  one  another  for  EEEV  business  unless  somehow  forced  to  do  so.  But  DOD  had  acquiesced 
in  the  formation  of  their  joint  venture,  UEA,  making  inter- firm  competition  essentially  a  dead 
issue. 

In  effect,  the  EEEV  program  was  back  in  the  traditional  (for  DOD)  acquisition  relationship 
of  a  single  buyer  and  sole  supplier.  In  search  of  lower  cost  it  now  turned  to  what  in  effect  was  a 
strategy  of  multi-year  procurement,  envisioning  a  five-year  contract  with  UEA  for  production  of 
eight  EV  booster  cores  per  year,  notionally  split  evenly  between  Atlas  and  Deltas.  Congress 
has  traditionally  cast  a  skeptical  eye  on  such  arrangements,  and  the  proposal  led  to  unwelcome 
attention  from  Capitol  Hill. 

The  Defense  Contract  Audit  Agency  (DCAA)  complained  that  there  was  nothing  to  audit 
because  the  EEEV  program  generated  so  little  cost  and  performance  data,  leading  to  GAO 
concerns  that  the  USAE  was  preparing  to  give  UEA  a  blank  check.  There  was  also  concern  that 
there  was  no  guarantee  that  the  number  of  satellites  to  be  launched  might  not  increase  enough  to 
absorb  eight  booster  cores  per  year.^"^  Over  the  following  year,  however,  the  Program  Office 
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implemented  the  multi-year  recommendations  well  enough  to  mollify  the  GAO. 
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In  March  2012,  EEEV  was  reinstated  as  an  MDAP  with  SAR  reporting  requirements. 
Since  cost  growth  had  exceeded  thresholds,  a  critical  Nunn-McCurdy  breach  was  reported.  Cost 
and  performance  reporting  was  strengthened  and  EVM  required. 

In  an  ADM  dated  27  November  2012  the  DAE  authorized  procurement  of  up  to  fifty 
booster  cores  over  five  years,  with  up  to  thirty-six  to  be  procured  as  a  block  from  UEA.  The 


possible  subsequent  certification  efforts  that  invariably  would  have  led  to  decisions  to  continue  DOD’s 
investment  in  the  EELV  program.  According  to  these  officials,  in  2006  the  program  was  facing  one  or  more 
possible  Nunn-McCurdy  unit  cost  breaches,  as  a  result  of  unplanned  cost  increases  resulting  from  the  change  in 
the  acquisition  strategy  and  a  decrease  in  the  demand  for  EELV  launches  (factors  include  satellite  program 
development  delays  and  cancellations,  and  operational  satellites  lasting  longer  than  anticipated).”  GAO,  Space 
Acquisitions,  GAO-08-1039,  20-21. 

A  light  or  medium  launch  requires  one  booster  core,  a  heavy  launch  requires  three. 

GAO,  Evolved  Expendable  Launch  Vehicle:  DOD  Needs  to  Ensure  New  Acquisition  Strategy  Is  Based  on 
Sufficient  Information,  GAO-11-641  (Washington,  DC:  GAO,  15  September  201 1). 

GAO,  Evolved  Expendable  Launch  Vehicle,  GAO-12-822. 

Research  conducted  for  this  paper  has  been  unable  to  determine,  without  more  extensive  investigation,  the 
events  or  decisions  that  led  to  this  action. 
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other  fourteen  are  to  be  open  to  eompetition.  The  eompetition  is  the  Falcon  9  developed  and 
produced  by  Space  Exploration  Technologies  Corp.  (SpaceX),  and  Orbital  Sciences  Corp.’s 
Antares.  Both  SpaceX  and  Orbital  Sciences  are  commercial  launch  providers  that  have 
developed  their  LVs  on  their  own.  They  claim  to  be  able  to  offer  prices  substantially  below  those 
of  ULA.  News  reports  indicate  that  the  USAF  has  begun  the  process  of  certifying  both 
companies  to  compete  for  LV  core  contracts. 

What  can  be  said  of  a  CAIV  program  that  has  so  far  had  more  than  300  percent  growth  in 
PAUC?  It  appears  that  the  EELV  program’s  commitment  to  treating  cost  as  an  independent 
variable  was  never  more  than  boiler-plate.  The  size  mix  of  boosters  needed  was  and  is 
determined  by  the  payloads  to  be  placed  in  orbit,  which  is  completely  outside  the  control  of  the 
program.  The  key  performance  parameter  is  launch  reliability,  and  it  makes  no  sense  to 
compromise  on  reliability  and  risk  destruction  of  expensive  payloads  that  might  be  critical  to 
national  security.  Thus,  there  was  virtually  no  room  for  significant  cost-performance  tradeoffs. 

In  light  of  these  facts,  the  decision  to  designate  EEEV  as  a  flagship  program  seems 
questionable.  By  claiming  a  commitment  to  hold  the  line  on  costs  in  a  program  wherein  making 
meaningful  performance  trades  was  not  a  realistic  possibility,  DOD  undercut  the  credibility  of 
CAIV  as  an  instrument  for  cost  control  in  general,  with  no  compensating  advantages. 

CAIV  aside,  it  can  be  concluded  that  DOD  proceeded  unwisely  in  its  cost-control  strategies 
for  EEEV.  In  the  context  of  expert  predictions  in  the  late  1990s  of  a  greatly  expanded  market  for 
launch  services,  it  was  reasonable  to  attempt  to  leverage  the  commercial  market  to  increase  EV 
volume  and  thus  cut  costs.  What  was  unreasonable  was  to  count  on  it  without  analyzing  the  risks 
and  without  putting  a  credible  fallback  strategy  in  place.  By  2000  it  was  clear  that  the  expanding 
commercial  market  was  a  chimera,  yet  EEEV  was  locked  into  commercial  competition  as  its  sole 
hope.  It  is  not  known  what  viable  fallback  strategies  might  have  existed,  but  to  have  failed  to 
recognize  that  one  might  be  needed  and  to  search  for  them,  diligently  and  imaginatively,  was 
imprudent. 

K.  Crusader 

Crusader  was  an  Army  program  to  develop  a  new,  highly  automated  155mm  self-propelled 
howitzer.  The  program  entered  the  PDRR  acquisition  phase  (now  known  as  the  Technology 
Maturation  and  Risk  Reduction  phase)  in  1994.  Crusader  consists  of  an  armored,  self-propelled 
howitzer  and  an  armored  resupply  vehicle.  The  objective  was  an  artillery  system  with  increased 
range  and  survivability  with  autonomous  operations.  Compared  to  existing  howitzers.  Crusader 
was  to  provide  increased  rate-of-fire,  hold  more  ammunition,  be  more  responsive  and  survivable 
on  the  battlefield,  with  reduced  manpower  requirements;  provide  increased  lethality;  be 
deployable  worldwide;  and  provide  forward  maintenance  and  employ  future  maintenance 
concepts.  The  ADM  directed  the  Army  to  plan  for  a  Milestone  II  Defense  Acquisition  Board 
(DAB)  or  equivalent  review,  incorporating  as  many  acquisition  reform  measures  as  practical. 


A-32 


The  PDRR  program  underwent  two  major  restrueturings.  The  first,  in  March  1996,  changed 
the  armament  system  from  a  liquid  propellant-based  system  to  a  solid  propellant-based  system  to 
reduce  development  risk.  The  second,  in  1999,  was  to  align  the  program  with  the  Army’s  then- 
current  vision  for  more  deployable  forces,  consistent  with  initiation  of  the  Future  Combat  System 
program.  This  required  Crusader  to  increase  deployability  while  retaining  all  key  performance 
parameters,  by  reducing  weight  and  volume  so  that  two  Crusader  vehicles  could  be  deployed  on 
a  single  C-5B  aircraft.  This  required  a  drastic  redesign,  requiring  almost  $1  billion  additional 
funds  for  the  development  program. 

After  the  September  11,  2001  attack  and  subsequent  military  engagement  in  Afghanistan,  a 
new  howitzer  seemed  less  of  a  priority.  Furthermore,  other  alternatives  to  achieve  higher 
lethality  in  indirect  fire  (namely  high  rate-of-fire  rocket  systems  and  guided  munitions)  could  be 
obtained  far  cheaper  than  Crusader,  and  Crusader’s  survivability  improvement  seemed  less 
important  in  the  type  of  conflicts  envisioned,  at  least  in  the  near  term.  Such  considerations  led 
Secretary  Rumsfeld  to  cancel  Crusader  in  May  2002. 

In  1996,  the  Army  named  Crusader  (along  with  ATAMCS/BAT)  a  CAIV  flagship  program. 
The  December  1998  SAR  reflected  the  program’s  status  as  a  CAIV  flagship,  including  the 
following  paragraph  in  the  Executive  Summary  section: 

Crusader  continued  to  apply  the  concepts  of  Cost  as  an  Independent  Variable 
(CAIV),  which  have  had  a  significant  impact  on  program  cost  estimates.  To  date, 
the  CAIV  efforts  on  Crusader  resulted  in  attaining  35%  of  the  Design-To-Unit- 
Rollaway-Costs  (DTURC)  reduction  goal.  Additional  cost  reduction  initiatives 
continue  to  be  identified.  There  are  currently  over  100  initiatives  in  various  stages 
of  evaluation.  The  initiatives  have  the  potential  to  further  reduce  cost  resulting  in 
a  high  level  of  confidence  in  meeting  Crusader’s  DTURC  goal.  Intensive 
management  efforts  this  year  have  resulted  in  (a)  defining  the  peacetime 
OPTEMPO  levels  that  represent  significant  reductions  from  the  current  system, 

(b)  building  a  Crusader  battalion  that  reflects  significant  reductions  in  personnel, 
and  (c)  utilizing  an  Auxiliary  Power  Unit  in  lieu  of  the  main  engine  resulting  in 
reduced  fuel  usage  and  maintenance.  These  results  are  all  indicative  of  a  proactive 
process  whereby  cost  is  a  key  criterion  in  all  design  and  programmatic  decisions. 

This,  on  its  face,  appears  to  reflect  a  serious  effort  to  control  and  reduce  costs,  both 
procurement  and  sustainment.  (No  statement  of  this  nature  appears  in  the  subsequent  SARs.) 
However,  the  statement  is  somewhat  misleading  because  the  35  percent  cost  reduction  cited 
occurred  well  before  designation  of  the  program  as  a  CAIV  flagship.  Unfortunately  the 


Prepared  Statement  of  the  Honorable  Donald  H.  Rumsfeld,  Secretary  of  Defense,  on  the  Crusader 
Recommendation  before  the  Senate  Committee  on  Armed  Services,  May  16,  2002. 

The  IDA  research  team  knows  this  because  the  1998  IDA  publication  on  the  CAIV  flagships  states  that  the  35 
percent  reduction  occurred  “prior  to  Milestone  I,”  which  was  in  November  1994,  well  before  the  CAIV 
initiative.  Danny  L.  Reed,  Report  on  the  1996-1997  Workshops  on  Cost  As  an  Independent  Variable  (CAIV) 
Flagship  Programs,  IDA  Document  D-2251  (Alexandria,  VA:  February  1998). 
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program’s  termination  eliminated  the  possibility  of  determining  the  ultimate  level  of  sueeess  in 
redueing  produetion  eosts.^^ 

Comparing  the  eosts  from  the  earliest  available  SAR  to  the  last  one  indieates  modest 
growth  in  the  produetion  estimate  eosts  (see  Table  A-8). 


Table  A-8.  Procurement  Cost  Estimates  for  Crusader  from  the  SAR  Acquisition  Program  Baselines 

(FY1995$  million) 

Qty.-Adjusted  Unit 
Unit  Cost  Cost 


Date 

Objective 

Threshold 

Qty. 

Objective 

Threshold 

Objective 

Threshold 

1/4/1995 

8868.2 

9311.6 

824 

10.8 

11.3 

10.8 

11.3 

10/23/1997 

8754.2 

9191.9 

815 

10.7 

11.3 

10.8 

11.3 

12/18/2000 

5430.5 

5702.0 

471 

11.5 

12.1 

11.3 

11.9 

Change  from  1995  to  2000 

5.3% 

5.3% 

These  eosts  are  reasonably  eonsistent  with  those  in  a  briefing  by  the  OSD  Offiee  of 
Program  Analysis  and  Evaluation  dated  April,  2002.  The  unit  eost  from  that  briefing  is  $14.7 
million,  given  in  then-year  dollars.  An  approximate  eonversion  to  FY  1995  dollars  yields  an 
estimate  of  $11.2  million.  Thus,  it  appears  that  there  was  little  or  no  growth  in  estimated 
proeurement  eosts  for  Crusader  between  1995  and  2002.  If  these  estimates  are  eorreet,  they 
would  presumably  indieate  some  sueeess  in  eontrolling  proeurement  eosts  (if  not  aehieving  a 
stated  CAIV  goal  of  a  25-50  pereent  reduction).  That  presumption,  however,  should  be  tempered 
by  the  fact  that  the  program  never  reached  Milestone  IT  Therefore,  an  independent  cost  estimate 
based  on  the  design  that  would  actually  enter  the  EMD  phase  was  never  made."^'^  It  is  more  than 
conceivable  that  the  cost  of  such  a  design  would  have  been  substantially  higher  than  the  PDRR 
estimate. 

L.  ATACMS/BAT 

The  second  Army  program  nominated  as  a  CAIV  flagship  was  ATACMS/BAT.  This 
program  met  much  the  same  fate  a  Crusader,  having  been  cancelled,  this  time  by  the  Army,  in 
late  2003.  Unlike  Crusader,  the  program  was  well  into  full-scale  development,  in  fact  in  low-rate 
initial  production  (TRIP),  prior  to  its  designation  as  a  CAIV  flagship.  Also  unlike  Crusader,  no 
recognition  of  the  program  as  a  CAIV  flagship  is  found  in  any  of  the  SARs. 

The  “Brilliant  Anti-Tank”  or  BAT  is  a  submunition  originally  designed  to  be  delivered  by 
the  TSSAM.  The  Army  withdrew  from  that  program  in  1993,  and  in  1994  the  entire  TSSAM 


Production  cost  estimates  prior  to  the  beginning  of  full-scale  development  are  subject  to  a  large  margin  of  error, 
since  there  is  yet  no  producible  design  on  which  to  base  a  constructive  estimate. 

The  last  SAR  in  December  2000  indicated  a  Milestone  II  (equivalent  to  today’s  Milestone  B)  in  April  2003. 
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program  was  cancelled,  due  primarily  to  eost  overruns.  After  the  Army’s  withdrawal  from 
TSSAM,  the  BAT  was  modified  to  be  delivered  by  the  Army  Taetieal  Missile  System 
(ATACMS).  The  program  passed  Milestone  II  in  May  1991. 

BAT  suffered  a  long  and  turbulent  development  program,  plagued  by  test  failures  of  the 
highly  ambitious  seeker,  whieh  was  designed  to  automatically  acquire  and  guide  the  missile  to 
home  on  vehieular  targets  moving  on  the  battlefield.  Soon  after  entry  into  TRIP,  the  program 
was  eaneelled.  Thus,  unlike  Crusader,  there  is  some  data  on  produetion  eosts.  Sinee  the  available 
SAR  database  from  the  Defense  Aequisition  Management  Information  Retrieval  System 
(DAMIR)  only  goes  baek  to  1997,  the  IDA  researeh  team  eould  not  obtain  eost  data  from 
Milestone  II.  However,  sinee  its  designation  as  a  CAIV  flagship  oeeurred  in  1996,  eost 
experienee  sinee  1997  is  applieable  to  the  evaluation  of  the  program  as  a  CAIV  initiative. 

Table  A-9  provides  the  reeord  of  ATACMS/BAT  PAUC  and  APUC  from  the  SARs.  In  the 
Deeember  2004  SAR,  the  quantity  was  essentially  redueed  to  those  that  had  been  proeured  at  the 
time  of  program  eaneellation.  Even  diseounting  the  last  entry,  it  ean  be  seen  that  by  1999  APUC 
had  grown  by  64  pereent  (1 19,000/19,700)  over  the  Deeember  1997  SAR  figure. 


Table  A-9.  ATACMS/BAT  Unit  Cost  History  from  SARs  (FY  1991$) 


SAR 

Date 

PAUC 

APUC 

Quantity 

Percent 
Change 
in  APUC 

Percent 
Change  in 
Quantity 

Quantity- 

Adjusted 

APUC 

Quantity- 
Adjusted 
%  Change 
in  APUC 

Dec  1997 

$135,000 

$72,000 

19,700 

$72,000 

Dec  1998 

$154,000 

$87,000 

19,554 

21% 

-1% 

$86,935 

21% 

Dec  1999 

$203,000 

$119,000 

15,707 

37% 

-19% 

$116,421 

34% 

Dec  2001 

$221,000 

$118,000 

16,089 

-1% 

10% 

$118,284 

2% 

Dec  2002 

$1,232,000 

$247,000 

1,262 

109% 

-92% 

$191,492 

62% 

Percent  Change  in  APUC  from  Original  Estimate 

166% 

The  available  evidenee  suggests  that,  if  any  CAIV-related  aetivities  took  plaee  in  the 
ATACMS/BAT  program,  they  were  not  suoeessful.  That  said,  it  ean  further  be  observed  that, 
while  the  SARs  doeument  eost  growth,  they  are  widely  reeognized  as  inadequate  in  providing 
explanations  of  the  eauses  of  eost  growth.  Although  eategorizations  of  and  reasons  for  eost 
ehanges  are  nominally  ineluded  in  the  “Cost  Varianee”  seetion  of  the  SAR,  it  is  widely 
reeognized  that  the  SAR  eategorizations  suffer  from  ineonsisteneies  and  ambiguities."^*  The 
explanations  of  proeurement  eost  growth  in  the  ATACMS/BAT  SARs  are  no  exeeption.  For 


Joseph  G.  Bolten,  Robert  S.  Leonard,  Mark  V.  Arena,  Obaid  Younossi,  and  Jerry  M.  Sollinger.  Sources  of 
Weapon  System  Cost  Growth:  Analysis  of  35  Major  Defense  Acquisition  Programs  (Santa  Monica,  CA:  RAND 
Corporation,  2008).  Available  on  RAND’s  website  at  http://www.rand.org/pubs/monographs/2008/RAND 
MG670.pdf. 


A-35 


example,  for  the  two  largest  proeurement  cost  increases,  occurring  in  the  December  1998  and 
December  1999  SARs,  the  following  explanations  are  offered  in  the  Cost  Variance  sections: 


Dec-98: 

Change  in  learning  curve  assumptions  due  to  re-phasing  of  annual  buy.  (Estimating)  -t-$21 .1 
million 

Refinement  of  estimate  to  reflect  BAT  hardware  updates.  (Estimating)  +$83.7  million 
Refinement  of  estimate  to  reflect  P3I  BAT  hardware  updates.  (Estimating)  +$1 18.5  million 
Refinement  of  estimate  due  to  BAT/P3I  BAT  program  restructure.  (Estimating)  +$63.3  million 
Dec-99: 

Revised  estimate  for  inclusion  of  submunition  budget  for  Block  IIA  (Estimating)  +$21 8.5  million 
Revised  estimate  to  reflect  negotiated  cost  of  production  contract.  (Estimating)  +$157.7  million 

Sources:  DOD,  SAR  RCS:  DD-A&T(Q&A)823-545  “ATACMS-BAT”  (Washington,  DC:  DOD,  December  1998),  44 
and  (December  1999),  42. 

Figure  A-11.  SAR  Explanations  for  Two  Largest  ATACMS/BAT  Cost  Increases 

With  the  exception  of  the  first,  these  explanations  are  opaque  as  to  the  underlying  causes. 

This  cursory  analysis  indicates  that  either  there  was  no  real  effort  to  implement  CAIV 
(suspected)  or  that  if  there  was,  it  was  unsuccessful.  In  the  latter  case,  the  reasons  cannot  be 
determined  without  access  to  and  analysis  of  more  detailed  historical  records  of  the  program. 
Obviously,  the  program  presented  serious  technological  challenges  (as  evidenced  by  the  repeated 
test  failures),  and  for  that  reason  alone,  was  probably  not  a  good  choice  for  a  CAIV  flagship. 
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Abbreviations 


AAAV 

Advanced  Amphibious  Assault  Vehicle 

ACV 

Amphibious  Combat  Vehicle 

ADM 

Acquisition  Decision  Memorandum 

AFCAIG 

Air  Eorce  Cost  Analysis  Improvement  Group 

AMPV 

Armored  Multi-Purpose  Vehicle 

AoA 

Analysis  of  Alternatives 

APB 

Acquisition  Program  Baseline 

APUC 

Average  Unit  Procurement  Cost 

ASD 

Assistant  Secretary  of  Defense 

AT&L 

Acquisition,  Technology  and  Eogistics 

ATACMS/BAT 

Army  Tactical  Missile  System/  Brilliant  Anti-Tank 

BAT 

Brilliant  Anti-Tank 

BBP 

Better  Buying  Power 

BY 

Base  Year 

CAIG 

Cost  Analysis  Improvement  Group 

CAIV 

Cost  as  an  Independent  Variable 

CAPE 

Cost  Assessment  and  Program  Evaluation 

CARD 

Cost  Analysis  Requirements  Description 

CCDR 

Combatant  Commander 

CD 

Concept  Definition 

CDD 

Capability  Development  Document 

CER 

Cost  Estimating  Relationship 

CMC 

Commandant  of  the  Marine  Corps 

CNO 

Chief  of  Naval  Operations 

CPIPT 

Cost/Performance  Integrated  Product  Team 

CSB 

Configuration  Steering  Board 

CTOE 

Conventional  Takeoff  and  Landing 

DAB 

Defense  Acquisition  Board 

DAE 

Defense  Acquisition  Executive 

DAES 

Defense  Acquisition  Executive  Summary 

D-l 


DAMIR 

DAPP 

DCAA 

DCAPE 

DCP 

DDR&E 

DOD 

DODD 

DODI 

DSP 

DEC 

DTURC 

EEEV 

EEV 

EMD 

ER 

EVM 

ECS 

EoV 

EPP 

PSD 

PTC 

PY 

GAO 

GCV 

GEO 

HEO 

HMDS 

HMMWV 

ICE 

IDA 

IPS 

IPT 

IR 

JASSM 


Defense  Aequisition  Management  Information  Retrieval 
System 

Defense  Aequisition  Pilot  Projeet 

Defense  Contraet  Audit  Ageney 

Direetor,  Cost  Assessment  and  Program  Evaluation 

Deeision  Coordinating  Paper 

Direetor,  Defense  Researeh  and  Engineering 

Department  of  Defense 

Department  of  Defense  Direetive 

Department  of  Defense  Instruction 

Defense  Support  Program 

Design  to  Cost 

Designed-to-Unit-Rollaway  Cost 

Evolved  Expendable  Paunch  Vehicle 

Expeditionary  Pighting  Vehicle 

Engineering  and  Manufacturing  Development 

Extended  Range 

Earned  Value  Management 

Puture  Combat  System 

Pamily  of  Vehicles 

Plagship  Pilot  Program 

Pull-scale  Development 

Pederal  Trade  Commission 

Piscal  Year 

U.S.  Government  Accountability  Office 

Ground  Combat  Vehicle 

Geosynchronous  Earth  Orbit 

Highly  Elliptical  Orbit 

Helmet-Mounted  Display  System 

High-Mobility  Multipurpose  Wheeled  Vehicle 

Independent  Cost  Estimate 

Institute  for  Defense  Analyses 

Initial  Paunch  Services 

Integrated  Product  Team 

Infrared 

Joint  Air-to-Surface  Standoff  Missile 
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JCIDS 

Joint  Capabilities  Integration  and  Development  System 

JDAM 

Joint  Direct  Attack  Munition 

JORD 

Joint  Operational  Requirements  Document 

JSF 

Joint  Strike  Eighter 

JTRS 

Joint  Tactical  Radio  System 

KPP 

Key  Performance  Parameter 

KSA 

Key  System  Attribute 

L&MR 

Eogistics  &  and  Materiel  Readiness 

LCC 

Eife-Cycle  Cost 

LCS 

Eittoral  Combat  Ship 

LEO 

Eow-earth  Orbit 

LM 

Eockheed  Martin 

LOAL 

Eock-on-After-Eaunch 

ERIE 

Eow-rate  Initial  Production 

EUH 

Eight  Utility  Helicopter 

EV 

Eaunch  Vehicle 

EVT 

Eow  Volume  Terminal 

MDA 

Milestone  Decision  Authority 

MDAP 

Major  Defense  Acquisition  Program 

MOD 

Material  Development  Decision 

MIDS 

Multifunctional  Information  Distribution  System 

MIECON 

Military  Construction 

MS 

Milestone 

MSA 

Material  Solution  Analysis 

NASA 

National  Aeronautics  and  Space  Administration 

NMM 

National  Mission  Model 

NRO 

National  Reconnaissance  Office 

NSS 

National  Security  Space 

O&M 

Operations  and  Maintenance 

O&S 

Operating  and  Support 

OCO 

Overseas  Contingency  Operation 

ODCAPE 

Office  of  the  Director  of  Cost  Assessment  and  Program 
Evaluation 

0MB 

Office  of  Management  and  Budget 

OPTEMPO 

Operating  Tempo 

OSD 

Office  of  the  Secretary  of  Defense 
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OSMIS 

Operating  and  Support  Management  Information  System 

OT&E 

Operational  Test  and  Evaluation 

OUSD(AT&L) 

Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Eogistics 

PA&E 

Program  Analysis  and  Evaluation 

PARCA 

Office  of  Performance  Assessments  and  Root  Cause  Analyses 

PAUC 

Program  Acquisition  Unit  Cost 

PDRR 

Program  Definition  and  Risk  Reduction 

PM 

Program  Manager 

POM 

Program  Objective  Memorandum 

PPBS 

Planning,  Programming  and  Budgeting  System 

R&D 

Research  and  Development 

R&M 

Reliability  and  Maintainability 

RAM-C 

Reliability,  Availability,  Maintainability,  and  Cost  Rationale 

RDT&E 

Research,  Development,  Test  and  Evaluation 

REP 

Request  for  Proposal 

ROI 

Return  on  Investment 

ROM 

Rough  Order  of  Magnitude 

SAE/AQ 

Office  of  the  Secretary  of  the  Air  Eorce  for  Acquisition 

SAR 

Selected  Acquisition  Report 

SBIRS 

Space-Based  Infrared  System 

SDD 

System  Development  and  Demonstration 

SECDEE 

Secretary  of  Defense 

SEEP 

Service-Eife  Extension  Program 

SRM 

Short  Range  Missile 

STOVE 

Short  Takeoff  Vertical  Eanding 

TMRR 

Technology  Maturation  and  Risk  Reduction 

TOA 

Total  Obligation  Authority 

TPP 

Total  Package  Procurement 

TSPR 

Total  Systems  Performance  Responsibility 

TSSAM 

Tri-Service  Standoff  Attack  Missile 

UEA 

United  Eaunch  Alliance 

URE 

Unit  Recurring  Elyaway 

USAE 

United  States  Air  Eorce 

USD(AT&E) 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Eogistics 
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USD  (C) 

USMC 

USN 

VAMOSC 

VECP 

VLO 

WIN-T 


Under  Secretary  of  Defense  (Comptroller) 

United  States  Marine  Corps 
United  States  Navy 

Visibility  and  Management  of  Operating  and  Support  Cost 

Value  Engineering  Change  Proposal 

Very  Eow  Observable 

Warfighter  Information  Network — Tactical 
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